3.

Software

3.1 Communications Engineering

The first challenge in the project consists of finding a way for an Arduino board to send
MIDI messages. This is a key challenge towards creating an early proof of concept. An
Arduino board can send information to, and receive from, a computer through the Serial
Port. The problem can then be defined more precisely, and it that of allowing the Serial Port
to communicate with a Digital Audio Workstation (DAW), which is the music production
software that will generate the sounds.

Figure 1 – Information path of a note

Research revealed several existing approaches to the given problem. Two of the most
promising solutions were tested thoroughly: the Hairless MIDI Serial and the DualMoco.
The Hairless MIDI Serial Bridge is a middle software between the Serial Port and the DAW,
it essentially reads what has been uploaded to the Serial Port and feeds it into the DAW.
Hairless was created by Angus Gratton for a workshop, as part of the Electrofringe 2011
festival in Newcastle [10]. The software was released with GNU Lesser General Public
License 2.1, meaning the software can be re-distributed and sold as long as the same
license is maintained, all code must be available to users. Hairless functions appropriately,
it was successful at allowing Serial Port to DAW communication, unfortunately, it has many
User Experience Design (UXD) problems:

 The software has to be started-up to use the keyboard

 The Serial Port has to be defined on start-up
 DAW has to be manually told to read the software as a MIDI instrument
 Updating any Arduino code whilst the Hairless MIDI Serial Bridge is running is not
possible, turning it off and on is lengthy process
 Figure 2 – Hairless MIDI Serial Bridge interface

If just after a week or so of testing the software didn’t feel fully comfortable and natural to
use, it would have eventually demined the User Experience (UX) of the kit. Furthermore,
the software would have to be included in the kit, either in a USB memory stick or as a link
to a specially built online server. Users would have to install it, meaning operative system
and system versions could become limiting factors. It is not impossible to do so, but it would
add unnecessary complications to the process of getting a keyboard to play.

DualMoco is an alternative firmware to the Arduino board´s native firmware [11]. It was
developed by Yoshitaka Kuwata a Japanese doctorate, and released under an Apache
License 2.0, in similar terms to the Hairless MIDI Serial Bridge’s license. With DualMoco an
Arduino board essentially becomes a MIDI instrument and it is read as such by DAWs. The
board can easily be switched back to its normal operating mode, in which it is detected as
an Arduino board by the computer. This is so that new code can be uploaded to the board
whenever needed. Testing proved DualMoco to have overcome many of the shortcomings
encountered in the Hairless MIDI serial.

Figure 3 –
Illustration of
DualMoco
framework
As soon as the keyboard is plugged into a computer, performing can begin, and no middle
software or lengthy setups are required. Switching between modes can be tricky to learn,
but it is very time efficient and reliable process. The main downside of this solution is the
need to change an Arduino’s firmware. Changing the firmware of an Arduino board is by no
means a simple process. Users would not be expected to be able to do this themselves, all
boards would be distributed with the DualMoco firmware pre-installed in house.

3.2 Code Development

DualMoco allows communication between the Serial Port and a DAW effectively.
Nevertheless, a piece of code is needed to generate the MIDI messages that will use this
connection. MIDI commands are strings of numbers. Hence, the fundamental duty of the
code is to detect key presses or releases and generate strings of numbers in the Serial
Port, following MIDI standards. The code written for this goal is original, meaning it was
developed from scratch. Code development began with a simple one note instrument and
was gradually expanded. This section showcases the most significant iterations in this
process.

3.3 Iteration 1

The first iteration of the code consists of a simple one key MIDI instrument. It allows a
switch to send a note on command when pressed and a note off command when released.
This iteration is shown in Figure 11. The Serial.println function sends the content of its
brackets to the Serial Port. In this case 144 is the decimal number equivalent to 10010000
in binary, which, as explained in the MIDI section, defines the note on command. 50
corresponds to the key D, and the value of 100 defines a velocity of 100. These three
consecutive Serial.println functions constitute a MIDI message, built of a status byte and
two data bytes. The code cannot be directly expanded to other keys at this stage. If similar
functions are replicated for other keys the resulting instrument would be monophonic.
Monophonic instruments (mono stands for “only” or “one” and phony stands for “sound”)
only one key can be pressed at any time, a different key can only be pressed once the first
is released. Although some synthesizers used to be monophonic [12] due to engineering
limitations, nowadays most keyboard instruments are polyphonic (poly meaning “many”,
and phonic meaning “sounds”).
Monophony is the side effect of the while function. Once the code begins processing the
overarching if function, it has to wait (due to the while function) for the button to be
released. During this time no other functions can be processed, meaning that if another key
is pressed in this time frame, nothing happens.

Figure 4 – First code

iteration

3.4 Iteration 2

To allow polyphony the while function is replaced by an else function in the second iteration
of the code. The code does not have to wait for a note to be released to move to the next
note.

Figure 5 – Uncomlpeted
second iteration of code

The if function is inside the void loop function, meaning it is repeated up to every 6.25e-8
seconds. The above approach malfunctions, because millions of note on messages (or
note off) are created every second, each time the loop restarts. To overcome this issue, the
code must track the status of the note in the previous loop cycle, and it should to be able to
decide, based on it, if a note on or note off message whether should be created or not. The
piece of code below applies this logic and successfully allows polyphony.
Figure 6 – Completed second code iteration

3.5 Iteration 3

So far velocity has been set at a constant value for all notes, but the keyboard must be
velocity sensitive. To achieve velocity sensitivity, the time difference between the actuation
of the two underlying buttons in each key is measured. The flowchart shown in the next
page showcases the logical process required to achieve these measurements.

Following this flowchart the actual code for one key is illustrated.

It can be noted from this section of the keyboard´s code, that the a key´s code must be
repeated or duplicated for all keys, variables also need to be renamed accordingly. This is
not programming best practice. This approach of duplicating the code for each key did in
fact cause reliability problems as soon as more than 5 keys were programmed. This is
likely because the processing power of the Arduino was not harnessed efficiently.

This issue was effectively resolved in the 4th iteration of the code by employing arrays.
 Figure 8 – Third code iteration

An equation that stands out from the above code is probably

Which in mathematical terms is equal to:

This equation is transforming the difference in time (in milliseconds) between button
actuations into a useable velocity value (0-127). Hence, the equation defines the velocity
curve of the instrument. Velocity curves relate velocity values to different key press forces
[13], the optimal velocity curve largely depends on the user. This is because some players
are used to playing harder than others.

The equation above provides another layer of personalization, as users can define exactly
what velocity curve best suits their needs.

The curve used in the 3rd coding iteration was determined through some methodical
subjective experimentation. The same key was pressed hard, normally and softly a
number of times. The average resulting time differences were recorded as 8ms (hard),
20ms (normally) and 150ms (softly). Expected velocity values for these keystrokes were
approximately 110, 85 and 40. With this data a graph could be drawn.

Finally, the lower limit of the graph was limited at 30, since values under 30 were rarely
ever needed for the line of musical work carried out. In the graph below the y axis
represents velocity and the x axis represents time difference.

Figure 9 –
Keyboard´s
velocity curve

A logarithmic curve avoids potential bugs and errors preventively by exploiting two
boundary conditions of the equation:

1. Time difference is always positive due to hardware constraints.

 2. As graph is inversely exponential, very long time differences will always give a
useable (0-127) velocity value.

3.6 Iteration 4

The 4th and final iteration as aforementioned, allowed a single function to operate all keys
using arrays, thus improving the code’s reliability and performance.
Figure 10 – Fourth code iteration