Agostini 2015

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 18

Andrea Agostini∗ and Daniele Ghisi† A Max Library for Musical


Conservatory of Cuneo
Via Roma 19
12100 Cuneo, Italy
Notation and

Institut de Recherche et Coordination
Acoustique/Musique Computer-Aided
1, Place Igor-Stravinsky
75004 Paris, France Composition
{andreaagostini, danieleghisi}@bachproject.net

Abstract: This article introduces a library of external objects for real-time computer-aided composition and musical
notation in Max. The library provides Max with a set of tools for the graphical representation of musical notation, ma-
nipulation of musical scores through a variety of approaches ranging from GUI interaction to constraint programming,
and sequencing. The library is oriented to real-time interaction, and is meant to interoperate easily with other processes
or devices controlled by Max, such as DSP tools, MIDI instruments, or generic hardware systems. These features
and design choices place our software at the intersection of various categories of musical software environments and
approaches, allowing it to help reduce the gap found between tools for sound-based, electroacoustic musical practices,
and for symbol-based, traditional composition. The library is called “bach: automated composer’s helper.”

A long-standing tradition has differentiated two operations on musical scores, such as transposition,
main fields of interest in computer music: digital augmentation, and playback. To some extent, all
signal processing (DSP) on the one hand, and these software systems can be effectively considered
treatment of symbolic musical data on the other. as having computer-aided composition capabilities,
The former has reached a high level of refinement which in some cases can be taken to a nontrivial
and public awareness in the last decades, with degree of complexity, for example, through the
professional-grade DSP algorithms nowadays widely use of certain advanced plug-ins in Finale, or
available and used in an increasingly broad set of through ManuScript, the scripting language of
devices and applications, including mobile phones Sibelius.
and embedded systems. The latter, on the other Still, in our own compositional activity, we feel
hand, seems confined to a small number of specialist there is a lack of systems and environments that
software systems such as OpenMusic (Agon 1998), would allow one to easily cross the boundaries be-
PWGL (Laurson and Kuuskankare 2002), Strasheela tween these categories. Musique mixte (music using
(Anders, Anagnostopoulou, and Alcorn 2005), or traditional instruments along with electroacoustic
Common Music (Taube 1991). Moreover, DSP is sounds) demands tools capable of representing and
used today to various extents in virtually any type processing complex instrumental scores associated
of musical production, as well as many nonmusical with rich acousmatic contents. Composition of
contexts. On the contrary, the use of systems for electroacoustic music is often performed within se-
symbolic processing is generally a prerogative of quencers such as ProTools or Logic, but this approach
certain specific styles of avant-garde experimental works best for music that is essentially based upon
music. The only exception to this relative isolation a compositional paradigm of montage. Music for
is represented by musical engraving systems such synthesizers most often needs a more flexible kind
as Finale or Sibelius. At their basis, they represent of score, as exemplified by Csound, which—in spite
possibly the simplest approach to the processing of its age and the crudeness of its graphical interface
of musical notation. Still, it might be simplistic and programming language—probably remains the
to consider them just as a strict translation of most widely used system of this kind. Improvisa-
the word-processing principle to music, because tional and performative musical practices benefit
they all implement some basic, yet significant, from the ability to represent musical scores in real
time, and having them interact with audio analyses
Computer Music Journal, 39:2, pp. 11–27, Summer 2015 and processes, MIDI streams, sensors, and actuators.
doi:10.1162/COMJ a 00296 Even a more traditional approach to instrumental

c 2015 Massachusetts Institute of Technology. music composition may be greatly enhanced by the

Agostini and Ghisi 11


immediacy of the real-time response of modern DSP Lisp, Clojure, Python, and more) was considered
systems and audio sequencers. Puckette (2004) and another interesting feature. In fact, all the most
Cont (2008b) have, on several occasions, tackled critical sections of bach are written in C or C++,
the need to bridge the gap between software tools which allow precise control of the graphical display,
for audio-based, electroacoustic musical practices flexible management of custom data structures,
and symbol-based, traditional composition. The and great computational speed. On the other hand,
computer-assisted orchestration project carried out interaction with languages other than C/C++ has
at the Institut de Recherche et Coordination Acous- proven more problematic than we had anticipated,
tique/Musique (IRCAM) is another approach to a because of the peculiar data structures introduced by
similar concern (Carpentier and Bresson 2010; Esling bach in the Max software ecosystem. This actually
and Agon 2010). leaves an open problem: Max, as a programming
Our attempt to give a partial response to these language, has a strong focus on building rich
various challenges is a library for the software Max interactions and user interfaces, but implementation
named BACH: Automated Composer’s Helper (here- of nontrivial algorithms for computation and data
after, simply “bach”), which we started developing processing is often less than straightforward because
in 2010 and which has since undergone continuous of the graphical programming paradigm itself.
development, also giving birth to a derived system Nonetheless, such nontrivial algorithms are often
named “cage.” necessary in the fields of computer-aided and
The library provides Max with graphical in- algorithmic composition, in order for composers to
terfaces for the display, editing, and playback of set up their own mechanisms for the manipulation
augmented musical scores, as well as more than 200 of symbolic musical data.
modules for processing these scores through vari-
ous low- and mid-level operations and paradigms,
ranging from basic list processing to constraint Overall Structure of the Bach Library
programming. Its tight integration within a popular
programming environment for audio and mixed me- The bach library is currently composed of roughly
dia, as well as its focus on real-time responsiveness, 110 Max externals (precompiled modules written
are meant to bridge the gap between sound-based and in C and C++) and 120 abstractions (editable
symbol-based systems. The bach library achieves Max patches that can be reused within other
this by taking advantage of existing technologies, patches and that are constituted of externals and
helping artists and media producers to avoid facing other abstractions). The choice of implementing a
the possibly steep learning curve of a totally new given operator as an external or as an abstraction
software system. is the result of several factors. First, there are
cases in which there is no actual choice: It is
generally impossible, or extremely cumbersome and
The Choice of Max inefficient, to implement as abstractions complex
user interfaces, operations requiring maximum
The choice of Max as the host environment for bach computational speed, primitive operators upon
was prompted by several considerations, the most custom data structures, or modules with certain
important being the already cited ease of integration problematic features such as the ability to change
with a multitude of processes and devices, including the number of their own inlets. Besides these
DSP, MIDI, visuals, and virtually any hardware specific cases, in the early versions of the library
system. Another important consideration was the approach was to make abstractions whenever
the maturity and stability of the Max graphical user possible, because this was a way to validate and
interface, and the power of its graphical API. The fact test the externals upon which the abstractions
that Max is scriptable through several programming themselves were based, to figure out whether
languages (C/C++, Java, JavaScript, Lua, Common new externals were needed, and to build a set

12 Computer Music Journal


Figure 1. An example of an elements at the root level
llll (a “Lisp-like linked (the topmost level in the
list”), displayed in tree tree representation). The
form via the bach.tree depth is the maximum
object. The length of such number of nested levels,
an llll is the number of including the root level.

of pedagogical patches showing how other bach


modules could be used within variously complex
contexts. This approach changed over time, and
more recently we have tended to build abstractions
only for very simple operations that can be carried
out with small patches. We even re-implemented
several modules that had previously been released
as abstractions, rewriting them as externals to
make them more efficient or to enhance them
with capabilities that were impossible, or very for “Lisp-like linked list”). An llll can contain all
complicated, to implement otherwise. the standard Max data types, i.e., integers, floats or
symbols, as well as rational numbers and other lllls,
to an arbitrary level of depth (see Figure 1). Most
Data Structures bach objects are designed to work upon lllls.
An example of an llll is 1 2 (3 4) 5 ((6/7
The first and main goal of the bach library is the 8/9) 1.0) 1.1 one point two, where pairs of
manipulation of musical scores, which are highly parentheses enclose sublists, i.e., lllls within lllls.
complex data structures. Max offers a limited choice On a side note, an llll can be used as an associative
of data structures, all of which are rather simple. The array by simply treating its first element as a key,
only exception is the dictionary, which is a sort of although this is less efficient than a hash-based data
ordered hash table that we did not find well-suited structure. This approach, similar to the association
to the kind of data representation we needed to list paradigm in Lisp, is indeed used quite often
implement. Indeed, we think that musical scores within bach. In most practical cases the cost of
can more easily be seen as collections of arbitrarily traversing the list to find the desired key can be
long sequential collections of data on various considered negligible, because this strategy is most
hierarchical levels (e.g., voices containing measures frequently used with relatively small sets of key-
containing chords), rather than as associative maps value pairs. Further, the object devoted to this task,
based upon symbolic keys. bach.keys, can be instructed to search the llll only
OpenMusic and PWGL typically represent a score at the specific depth levels at which the keys are
as an opaque object, accessible only through a set expected.
of predefined methods, and consisting of simpler Readers acquainted with the Lisp programming
opaque objects, such as voices, notes, or rests—thus language will notice that an llll, in contrast to a Lisp
reflecting the hierarchical paradigm described above. list, is not itself enclosed in a pair of parentheses.
Although elegant from a purely theoretical point of This difference is meant to ease the relation with
view, we find this approach to be somewhat limiting the Max environment, as in this way any standard
with respect to the user’s freedom of performing Max message, such as foo bar 1 2 3, can be
arbitrary operations that we might not have foreseen. considered as an llll containing no sublists. On the
Our choice was therefore to provide Max with a new other hand, this prevents objects from exchanging
data structure that is powerful enough to represent a Max- or Lisp-style atoms, since, for example, the
score, but simple enough to be manipulated through message foo will, in any case, be treated by bach
a generic and, as far as possible, a “language- objects as an llll containing only a single element.
agnostic” set of tools. At the same time, we wanted This brings a substantial simplification with respect
this data structure to allow easy exchange of data to the data-type system of both Max and Lisp.
with the major Lisp-based systems, such as the Moreover, lllls are always passed by value between
aforementioned OpenMusic and PWGL. Hence, we objects, through a combination of deep copying and
chose to implement a tree structure inspired by the reference counting techniques that is completely
Lisp list, called llll (pronounced el-el-el-el, standing transparent to the user and mimics the behavior

Agostini and Ghisi 13


of standard Max messages. This paradigm has the specifies whether the lllls must be sent out their
advantage of being more easily predictable by the outlets in native or text format. On the other hand,
user, who never has to worry about cloning or the all the bach objects can accept lllls in both formats,
difference between destructive and nondestructive and automatically interpret them correctly.
operations. Extensive testing showed that the
performance penalty caused by deep copying is
generally negligible in common, real-life usage Lambda Loops
scenarios.
Some bach objects and abstractions consistently
implement a specific mechanism called a lambda
Data Representations loop, whose goal is providing a sort of replacement
for lambda functions within the Max environment,
Max messages have an intrinsic length limitation because concepts such as anonymous functions or
which, as of Max 7, is set to 32,767 elements. lambda calculus would be completely extraneous to
Although comfortably sufficient for most uses, this the overall programming philosophy and paradigm
limitation can prove to be extremely restrictive of Max. It should therefore be made clear that the
when dealing with musical scores, as they can be “lambda” name is not used literally in this context,
composed of a very large number of complex items, but rather as an allusion and a hint for the user.
each represented by several constituting elements. One typical practice in computing is to combine
Moreover, passing lllls as regular Max messages, two operations in order to specify a given subclass
with sub-lists being expressed through pairs of of a more generic problem. For example, one might
parentheses in a sequential array of elements, want to combine a modulo 12 operation with a
requires parsing these arrays and converting them generic sorting algorithm in order to sort MIDI
into an actual tree structure in order for them pitches according to their pitch classes. Different
to be operated upon. This mechanism is indeed programming languages have various constructs
implemented in all the bach objects, and is essential for this, and Lisp’s lambda functions are probably
because it allows standard Max objects to access among the most elegant and powerful. Lambda
the contents of lllls, provided that they are not too loops are not an actual language feature, but rather
long. This is called the llll text format. But the a design pattern taking advantage of Max’s general
preferred passing mechanism for lllls, whenever callback-based operation model. In short, a lambda
they are exchanged between bach modules, is the loop is a patching configuration in which one or
so-called native format. In native format, when more dedicated outlets of an object or abstraction
viewed through a regular Max object such as print, output data iteratively, in the usual form of native
lllls appear as opaque messages consisting of the or text lllls, to a patch section. This patch section
bach.llll message followed by an integer, which calculates a result to be returned to a dedicated inlet
is the key for an internally maintained table that of the starting object. The result for each iteration
bach objects can access, and that associates these is typically a modification of the original data or
numeric keys with the actual llll data structure. In a number, which can represent a qualitative value
this way, bach objects can exchange lllls of arbitrary (such as in bach.constraints, described in the
length, and the llll exchange itself can happen at a section Constraint Programming) or a Boolean. The
much greater speed than if it required the parsing modification must be immediately fed back into
described earlier. Although standard Max objects the object originating the lambda loop, because
cannot access the contents of an llll in native the subsequent course of the iteration may depend
format, they can control its flow. Thus, objects such on it. In bach terminology, the outlets and inlets
as gate, trigger, and send can correctly deal dedicated to these transactions are called lambda
with native-format lllls. All the bach objects that outlets and lambda inlets, respectively, and are
can produce lllls accept the attribute “out,” which always the rightmost outlets and inlets of the

14 Computer Music Journal


Figure 2. Two examples of of the way the atoi object classes. Pairs of elements remainder modulo 12 of
lambda loops. In the top handles characters beyond of the incoming llll are the left element is less
patch, we modify each the traditional, seven-bit sent through the two than or equal to the
element of the incoming ASCII character set, the rightmost (lambda) outlets remainder modulo 12 of
llll by reversing the patch would need of bach.sort, and before the right element.
symbol. Symbols are sent, additional logic to handle using the standard Subsequently, bach.sort
one by one, through the those characters.) In the ordering given by sends out its outlet a
right, “lambda” outlet of bottom patch, we sort an bach.<=, we apply a list of MIDI pitches
bach.mapelem, reversed incoming list of MIDI modulo operation on both corresponding to the notes
via the itoa, zl rev, and pitches (corresponding to of them. This tells C4, C5, C5, D4, F3,
atoi objects, and fed back the notes C4, D4, C5, F3, bach.sort that the two F6, G6.
into the lambda inlet of C5, F6, and G6) elements will be in the
bach.mapelem. (Because according to their pitch correct order if the

tion of time. The bach.score object implements


a traditional, measured representation including
notions of tempo, time signature, measures, and
rests, whereas bach.roll always represents time
in milliseconds, and does not support the aforemen-
tioned notions. This distinction is motivated by
several considerations. These include the fact that
the criteria for horizontal spacing with respect to
time are radically different (strictly proportional for
bach.roll, based upon more complex relationships
and spacing tables for bach.score). Furthermore,
as explained in the following, the data structures
underlying the graphical representations are dif-
ferent. The bach.roll object can generally be
considered as the simpler one and—besides actually
displaying proportionally notated music—is the
usual choice for representing pitch structures whose
temporal parameters are irrelevant or nonexistent.
The bach.roll and bach.score objects share the
vast majority of their features, however—the most
object or abstraction. It should be remarked that important of these being the fact that every aspect
on no occasion can a lambda loop cause a stack of their contents can be edited both by messages and
overflow, because lambda inlets are always “cold” with mouse-and-keyboard interaction, thus allow-
(i.e., following Max terminology, they do not trigger ing them to be used in a variety of scenarios, from
an immediate result). completely interactive to completely algorithmic. In
The example of sorting by pitch class would thus every case, the score display is always immediately
be implemented by using the bach.% and bach.<= updated.
modules, in combination with the bach.sort Generally speaking, the most basic element in
object, as shown in Figure 2. both objects is the note. Notes belong to chords
(although in bach.score even noteless chords can
exist, and are treated as rests). In bach.roll chords
Bach Modules by Family belong directly to voices, whereas in bach.score
chords belong to measures, which in turn belong
The modules composing the bach libraries can be to voices. In bach.roll each chord has an onset
subdivided into a number of families, according to (a temporal starting point in the score), whereas
their purpose and behavior. in bach.score each chord has a duration (chord
onsets can be inferred from the sequence of chord
durations and the tempo). In both bach.roll and
Score Editors and Sequencers
bach.score each note has a pitch (expressed in
At the forefront of bach are the bach.roll and MIDI cents, that is, hundredths of a semitone above
bach.score objects. They are essentially two score MIDI pitch 0) and a MIDI velocity. In bach.roll
editors, whose main difference is the representa- each note in a chord also has an independent

Agostini and Ghisi 15


duration, and in bach.score each note also carries score item can be queried at any time. By taking
an additional flag representing tie information. advantage of these features, virtually any kind of
Besides these usual properties, notes can contain graphical music representation can be displayed
a large amount of metadata of various types: through bach.roll or bach.score.
numbers, text, lists, breakpoint functions, file In addition, both bach.roll and bach.score
names, and DSP parameters such as filter definitions support the placement of markers and the assign-
or spatialization trajectories. These are organized ment of symbolic tags to each score item, allowing
as structures called slots. Slot data can be input tag-based selection, grouping, and editing of musical
via the graphical interface as well as via message elements.
interaction. The latter approach makes both objects The bach.roll and bach.score objects can
capable of recording virtually any stream of data exchange whole scores with other objects in the
in real time, as long as it is associated with note form of lllls, whose structures essentially reflect the
events. structure of the internal representation described
Both bach.score and bach.roll can play back above. After a header section containing a set of
the scores they contain, and each time a note is global information, such as clefs, keys, or the data
encountered all its slot data is sent, along with types contained in the slots, the actual body of
the standard set of note properties, through the the score follows (see Figure 3). The body of the
object outlets. This feature makes bach.roll and llll representing a score contained in a bach.roll
bach.score two extremely flexible sequencers, object is structured in sublists, each representing
capable of driving any possible process or device a voice; each voice is structured in sublists, each
that can communicate with Max in real time. Real- representing a chord. Each chord sublist contains a
time playback, by which events are automatically global onset and a sublist of notes; each note contains
scheduled in time in Max’s scheduler thread, is information about its pitch, velocity, duration,
activated by the play message and ends either plus some optional specifications, structured in
when a stop message is received or when there sublists themselves, determining the enharmonic
is no more score content to be played. Several properties, the glissando breakpoint function, and all
options are available, including the ability to the metadata associated with the note. The body of a
define a looping region, to play a rectangular score bach.score llll is potentially much more complex
portion, or to play the currently selected items than a bach.roll llll because of the intricacies
(which may be noncontiguous). There is also the of the traditional rhythmic representation, such as
so-called offline playback, a kind of time-wise, grace notes, nested tuplets, or different beaming
virtually instantaneous dump (as opposed to the options. Still, for simple cases it closely resembles
standard voice-wise dump, or the real-time, time- that of bach.roll, except for the fact that chords
wise playback) of the musical score. This can be are contained in measures contained in voices,
useful for processes depending on the temporal notes may have starting or ending ties, duration is
relations among the various voices. Typical usage a property of chords rather than notes, and some
examples include synthesizers or DSP processes other minor differences. Users can algorithmically
requiring parameters that are too numerous or too edit a score either by working upon its complete
complex to be easily controlled through a standard llll representation, or by separately addressing its
MIDI sequencer. parameters such as pitches, duration, or slot content.
The graphical appearance of scores is highly Both bach.roll and bach.score support
customizable through the association of note data microtones of arbitrary precision: semitones, quarter
and metadata with graphical parameters, or—in tones, and eighth tones are represented by their
extreme cases—through the layering of Max’s standard symbols; for all the other subdivisions,
standard drawing tools, such as the lcd object, on the deviation from the reference “white key” pitch
top of bach.roll or bach.score. This is made can be represented as a fraction or in MIDI cents
possible by the fact that the pixel position of each (see Figure 4). Moreover, the scores contained in

16 Computer Music Journal


Figure 3. The score of a soloed. The tree shows MIDI cents, a duration of to recognize the second
simple bach.roll shown that the score has two 672 msec, a MIDI velocity voice, containing two
in tree form. The four voices, corresponding to of 100, and its flag set to single-note chords, in the
levels in the tree the two sublists at the top zero. The second note (the tree representation. This
representation correspond level. The first voice higher one) has a pitch of figure does not show the
to voices, chords, notes, contains one single chord 7,300 MIDI cents, a header section, containing
and parameters of notes. and a 0 flag, meaning that duration of 572 msec, a data such as clefs, keys, or
The last element of each the voice itself has no MIDI velocity of 88, and markers. The full score
level (except the top level) special properties; the its flag is set to two, data, including header,
is a flag determining, chord has an onset of 150 meaning that the note is can be obtained by
among the other things, msec, two notes, and the muted (on a color display substituting the dump
whether the corresponding flag set to zero. The first the note will be shown in body message with a
voice, chord, or note is note (that is, the lower blue to indicate the mute). simple dump message.
either locked, muted, or one) has a pitch of 6,300 Correspondingly, it is easy

is, a two-dimensional lattice diagram representing


the space of pitches generated by two arbitrary
diatonic intervals.

Generic Data-Processing Objects


A large subset of the bach library is aimed at
processing lllls. Several of these modules are inspired
either by their Lisp counterparts or by standard Max
list processing objects, such as zl. In contrast to
standard Max objects, though, not only is the llll
structure correctly recognized, but it is possible,
wherever meaningful, to perform each operation at
all the depth levels of the incoming llll, or at specific
depths. This is generally accomplished through
the attributes “mindepth” and “maxdepth,” which
indicate the depth of the outermost and innermost
elements, respectively, that should be affected
by the operation. These depths can be calculated
bach objects can be imported and exported in starting from either the root of the llll, or the deepest
several file formats, such as standard MIDI files, level of each of its branches. In the latter case, the
MusicXML, LilyPond files, and PDF (actually depth is entered as a negative number. Among the
rendered by LilyPond, provided that it is installed). other things, this allows the user to specify that
Scores also can be directly imported and exported to a given operation must be performed at all levels,
and from OpenMusic and PWGL. The bach library regardless of the overall depth of the llll, by setting
is also able to read and write Sound Description the attributes respectively to 1 (that is, the root
Interchange Format (SDIF) files, but because of the level) and −1 (that is, the innermost level of each
complexity and flexibility of the SDIF specification branch of the llll). See Figure 5 for an example.
their contents are not directly imported by the score This subset of bach can be further divided into
editors. This task can, however, be accomplished several categories:
through a family of abstractions belonging to the
cage library. 1. Basic operations (rotation, reversal, flatten-
ing, splitting, chaining, etc.)
2. Retrieval of general information such as the
Other User Interface Objects for
length or the depth of an llll, or the data
Musical Representation
types it contains
Two other user interface objects allow alternative 3. Insertion, substitution, and retrieval of
musical representations. The bach.circle object elements based upon positional or qualitative
is a classical clock diagram, useful to represent pitch criteria
classes or rhythmic positions on a looped grid. The 4. Set operations (union, intersection, differ-
object bach.tonnetz implements a Tonnetz, that ence, symmetric difference)

Agostini and Ghisi 17


Figure 4. Three different deviation from E4, cents, or 1/200 of a whole
bach.roll objects approximated to the tone (c). The internal
displaying the same note closest eighth tone, using a representation of the pitch
with pitch 6322.2 MIDI conventional accidental is never approximated
cents but with different notation (a); as a fraction, unless the user explicitly
graphical approximations approximated to 1/50 of a asks to snap it to the
set by the attribute whole tone (i.e, four cents) microtonal grid.
“tonedivision.” The (b); as a difference in

5. Iteration over the elements constituting an Rhythmic Quantization


llll, with several options allowing control of
bach.quantize is a highly customizable tool
the behavior with respect to the structure
designed to solve rhythmic quantization problems.
and the contents of a single llll or several
In the simplest case, bach.quantize receives
lllls to be iterated against each other
bach.roll’s proportional scores and converts them
6. The collection of elements received sequen-
into a bach.score, taking as an additional input
tially, with the possibility of defining and
either some direct information about tempo and
navigating the structure of the llll that is
meter, or a set of markers from which to infer the
being built
metric information. Such markers are typically
These two last categories, when combined, allow the contained in the original bach.roll.
implementation of virtually every process involving Once the meter and tempo are known,
element-by-element data manipulation. bach.quantize cuts the proportional score into
chunks, essentially corresponding to the beat units,
and tries to achieve a rhythmic quantization inside
Specialized Data-Processing Objects
each chunk. This “box-oriented” approach imple-
Other modules can operate only on appropriately ments the idea that the quantization algorithm must
structured lllls. For example, the bach.expr object assure synchronicities at least on an important set
can evaluate mathematical expressions on lllls of time instants (the beats by default, or some other
composed solely of numbers, including rationals. custom set). Inside each box, bach.quantize runs a
There is also a family of modules that can perform greedy backtracking algorithm that tries to optimize
matrix operations on lllls that have a matrix the onset error at each step. Each output duration
structure. Other objects will only be able to work, is chosen from linear combinations of a basic set of
or at least to return meaningful results, if they are durations, called minimal units and expressed as an
provided with lllls representing musical content attribute of the object. For instance, with minimal
of various kinds. For example, bach.n2mc will units of 1/16 and 1/12, any duration in the resulting
convert note names such as C4 or Re#2 into the score will be either a multiple of a sixteenth note, a
corresponding values in MIDI cents. Of course, multiple of a triplet eighth note, or, optionally, the
the GUI objects themselves belong to this broad sum of the two. The default behavior, however, is
family, as they can only operate upon lllls structured to allow only one of the minimal units inside each
according to the syntax specific to each of them, box. This implies that, for each box, all the resulting
such as the bach.roll and bach.score syntaxes durations will be multiples of just one quantization
described earlier. Some automatic conversions unit (see Figure 6). This keeps the tuplet writing
are made: For example, between numeric types in the output simple and also allows the algorithm
(integers, floating point, and rational fractions) to run faster. The user can, however, change this
where necessary and meaningful, and between note behavior at any time, allowing mixed combinations
names and the corresponding MIDI cent values (but of minimal units.
only for the limited number of modules dedicated to The attribute “Minimal Units” is probably the
notation). most important parameter that the user might

18 Computer Music Journal


Figure 5. The reversal affected by the reversal Figure 6. A simple markers. The result will
operation performed by (lowercase letters still quantization of a use only multiples of an
bach.rev is limited to the precede uppercase letters, bach.roll into a eighth note, or multiples of
second and third levels of and letter–number pairs bach.score, using the a triplet eighth (i.e., one
depth. The root level and are left unaltered). metric information twelfth), as set by the
the fourth level are not inferred from the time attribute “Minimal
signature and barline Units.”

desired result, rather than in terms of the steps


necessary to generate it. Constraints programming
want to modify. A configuration with many small is a programming paradigm widely used for this
minimal units would probably lead to an accurate, kind of problem, and most systems for computer-
but slow, quantization process. With fewer, larger aided composition provide tools implementing it
minimal units, the result would be more approxi- in some form, e.g., PWConstraints (Laurson 1993)
mative, but the computation would be faster. The in PatchWork, PMC (Anders and Miranda 2011) in
minimal units can also be assigned dynamically, PWGL, OMClouds (Truchet and Codognet 2004)
depending on the density of events of each chunk. or OMCS and Situation (Bonnet and Rueda 1998)
For instance, one might want to quantize fast, dense in OpenMusic. At the extreme of this range, some
musical events using more-detailed rhythmic subdi- software systems are entirely devoted to musical
visions; slower, sparser gestures could use a coarser constraint programming, such as Strasheela (Anders,
quantization to yield simpler rhythmic figures. Anagnostopoulou, and Alcorn 2005).
As previously indicated, bach.quantize is a The bach library contains a simple constraint-
highly customizable tool, featuring a large number solving object, called bach.constraints
of attributes. For example, the quantization is, by (see Figure 7). The main design feature of
default, calculated voice by voice. This means that bach.constraints is that it does not require
synchronous events across different voices might any textual coding proper. The object receives the
not be preserved during the process. This behavior domains of the variables and a list of the relations
can be changed by activating the attribute “Preserve between each variable (represented by a numeric
Synchronicities.” index) on the one hand, and the rules that directly
To give another example, bach.quantize by affect its labeling (rules are represented by arbitrary
default renders as grace notes those rhythmic events symbolic names), on the other. During the computa-
that are too small to be represented normally. The tion, the object iteratively sends proposals of partial
attribute “Small Notes Policy” lets the user override assignments of values taken from the domains to
this behavior. variables out of its lambda outlet. Each proposal is
evaluated in a lambda loop according to one named
rule. The result of the evaluation is subsequently
Constraint Programming
returned to the lambda inlet of bach.constraints.
It is often convenient to state musical formal- Two constraint-solving engines are currently
ization problems in terms of the qualities of the available in the bach.constraints object. One

Agostini and Ghisi 19


Figure 7. A simple example the rightmost inlet and constraint acting on the re-injected in the lambda
of constraint solving via outlet implement the three variables at the same inlet. In this example, the
bach.constraints. The constraint rules via a time. Combinations of search is performed by the
leftmost inlet receives the lambda loop. We define a values, associated with the backtracking engine, but
domain for each of the “noncons” constraint name of the rule against the hill-climbing engine
three variables of the acting on pairs of which they have to be might be invoked by
problem (specified by variables, and tested, are sent out of the setting the attribute
sublists of the main llll); corresponding to the lambda outlet, routed “weak” to one.
the middle inlet defines request that no variables according to the rule
the names and the scopes be consecutive numbers; name, and evaluated; the
of the constraints; finally, and a “pythagoras” result of each evaluation is
case, the result of the evaluation that is fed back
into the lambda loop is not a Boolean, but rather a
numeric value indicating the quality of the partial
assignment being considered. Second, in most cases
it can solve classic constraint-satisfaction problems
much more efficiently than the backtracking solver,
as long as a single solution is required. On the other
hand, because the hill-climbing algorithm has only
a very local notion of the topology of the search
space, it is in principle impossible for it to find all
the solutions of a problem, and it is possible that
no exact solution is ever found if the problem is a
is based on an optimized backtracking algorithm, particularly hard one.
the other is based on a hill-climbing algorithm with As stated earlier, all the rules for
taboos. bach.constraints are expressed outside the
The first of these is useful for solving classic object itself, in the lambda loop: This means that
constraint-satisfaction problems and only accepts no optimization based upon the specifics of the
Boolean values (represented as zero or one) in particular problem is possible. Moreover, the com-
the lambda inlet. Therefore, it can only return munication between objects in a Max patch is
solutions solving the problem completely, and it relatively slow, and even slower if the data to
has the ability to find all the problem solutions if be passed are lllls, because of the complexity of
required. In principle, the backtracking algorithm is the data structure itself. Although a performance
guaranteed to find a solution if it exists, although in penalty is not noticeable in most practical cases,
practice the time necessary for the computation can it often is with bach.constraints because of
be extremely long for large problems. A feature that the great number of lllls that potentially traverse
is currently under development, and that is only the lambda loop. This makes bach.constraints
applicable to the backtracking engine, is the ability impractical if large problems need to be solved,
to perform searches on relation domains, i.e., a sort especially when compared with engines based on
of multilevel search in which the actual variable textual coding, such as Gecode (www.gecode.org),
domains at one level depend on the solution of a which has been interfaced to OpenMusic and Max
constraint-satisfaction problem at the level below. by Lemouton (2011). As an initial strategy to im-
This approach can be useful for representing, for prove usability, if not efficiency, the lambda loop
example, harmonic problems in which one set of of bach.constraints can run in a custom thread
rules governs a sequence of harmonic classes, and if the “parallel” attribute is set to 1. Indeed, this is
another set of rules governs how pitches of different explicitly forbidden by the Max documentation for
harmonies interact with each other. Tonal harmony developers, as it can cause serious stability problems
is one classic case of this kind of problem. to the whole Max environment. On the other hand,
The second constraint-solving engine, hill climb- if some simple precautions are taken (such as not
ing, has two main fields of application. First, it can deleting or modifying objects and connections in the
solve weight-based optimization problems, in which, lambda loop, nor saving or closing the patch while
instead of searching for exact solutions, the goal is to the process is running), such a parallelized behavior
find “reasonably good” solutions (i.e., solutions for appears to be perfectly reliable, and it can be very
which it is reasonable to assume that they are among useful, allowing Max to perform other operations
the best possible within the problem space). In this while long searches are performed.

20 Computer Music Journal


Figure 8. An example of bach.roll at the bottom
real-time score granulation of the patch. The resulting
generated by the bach.roll is also being
cage.granulate module. played. The grains have a
The grains are extracted distance of 500 msec and a
from the loop region of the duration between 500 and
top bach.roll and added 2000 msec.
in real time to the

We are currently considering other, more specific communication between the different modules
options to improve efficiency, at least for some happens mostly by means of lllls.
categories of problems. At the same time, we think
that the nontextual approach, consistent with the
general Max syntax and the lambda-loop design The Cage Library
pattern, is well suited for quick prototyping, in
particular for users without a strong traditional Currently, the only project built starting from bach
programming background. is the cage library (Agostini, Daubresse, and Ghisi
2014). Whereas most bach modules are tools for low-
level or conceptually basic operations, cage is aimed
The Bach Family at easing higher-level manipulation of symbolic
musical data (see, for instance, Figure 8) and solving
Possibly the most ambitious project linked to bach, typical problems in computer aided-composition.
for the short and medium term, involves situating These include generation of pitches, generation and
the bach library inside a wider family of tools for processing of melodic profiles, symbolic processes
real-time computer-aided composition in Max. The inspired by digital signal processing, harmonic
idea is that bach might constitute the core of the and rhythmic interpolations, cellular automata,
family, and each additional library would expand L-systems, tools for musical set theory, and tools
bach to deepen some aspect of interactive computer- for score generation and handling. The cage library,
aided composition. All the bach family will inherit whose development is supported by the Haute École
from bach the basic principle of allowing computer- de Musique in Geneva, has a chiefly pedagogical
aided composition processes in real time, as well function, because all the modules in the library are
as some standard mechanisms and paradigms, such abstractions, lending themselves to easy analysis
as the lambda-loop design pattern, or the fact that and modification.

Agostini and Ghisi 21


Figure 9. In Andrea bach.roll object. In this instructions to be routed bach.roll object for
Agostini’s music for the example, note pitches are to the synthesizers. The the boundaries of the
film Wunder der ignored by the synthesis synthesis timeline is displayed section of
Schöpfung, the timeline for process. Notes are, thus, frame-synchronized with the score.
a complex, custom sound only containers for the video file of the film
synthesis system is numeric data, breakpoint during both editing and
entirely managed by one functions, and textual playback, by querying the

Perspectives completely focused on reactive, nonstandard GUI


interfaces for real-time computer-aided composi-
Imagining and sketching the future members of the tion, is currently under study. Among other things,
bach family is a thrilling and creative task, although this library might also feature some high-level,
such members still belong to the realm of planned graphical tools to facilitate a semi-automatic, “in-
possibilities. telligent,” user-friendly quantization, also by taking
For one thing, by design bach only implements inspiration from the OM-Kant library for Open-
essentially traditional representations of music. Music (Agon et al. 1994), which is now no longer
This is both its strength and its limitation. It is supported.
a strength, inasmuch as it allows bach to be a Another member of the bach family, which we
general-purpose, highly adaptable tool. It is a lim- are considering as a potential future project, might
itation since we think that, as a research project contain DSP objects linking sound generation and
into performative paradigms in computer-aided processing with symbolic representations. It might
composition, it should also cover experiments feature, for instance, real-time audio analysis tools
with new interfaces alongside more traditional designed and implemented to communicate quickly
ones. To fill this gap, a library named “dada,” and easily with bach.roll and bach.score.

22 Computer Music Journal


Figure 10. A portion of the represent electronic
electronic score for gestures in the score.
Daniele Ghisi’s Chansons, (On-screen, the elements
consisting of a bach.roll are differently colored.)
object, whose notes are Some metadata, such as
grouped into larger amplitude envelopes and
structures to better filters, are also displayed.

Relation to Other Software Furthermore, to facilitate the exchange of musi-


cal scores, a direct import and export mechanism
The bach library has a strong relation to differ- has been developed from and to OpenMusic and
ent families of software. For one thing, its design PWGL. (In the case of the latter, the conversion also
is explicitly inspired by the family of Lisp-based preserves most of the metadata contained in the
computer-aided composition software originating note slots.) Apart from these similarities, however,
with PatchWork (Laurson and Duthen 1989), and all the modules of bach are designed to operate in
in particular OpenMusic (Agon 1998; Assayag et al. the dataflow, real-time paradigm of Max, which
1999) and PWGL (Laurson and Kuuskankare 2002). is substantially different that of OpenMusic and
The “reductionistic” operational and representa- PWGL, and many of the possibilities that the bach
tional paradigm of bach, based upon the hierarchical paradigm offers are simply not achievable in the
arrangement of the different parameters of a score aforementioned environments (see, for instance,
and their individual manipulation, as well as some Figure 8). On the other hand, there are still domains
aspects of the system ergonomics, is directly bor- in which those environments largely outscore bach:
rowed from such environments. For example, the In particular, processing-intensive computations,
arrangement of the inlets and outlets of bach.roll such as combinatorial processes, and operations
and bach.score mostly matches that of the Open- that are inherently non-real-time, such as batch
Music objects chord-seq and voice, with the left- processing or generation of sound files.
most inlet/outlet pair handling global information, The bach library shares with projects such as
and a set of inlet/outlet pairs handling information psw.uscore (www.peterswinnen.be), MaxScore
about separate musical parameters (pitch, duration, (Didkovsky and Hajdu 2008), and InScore (Fober,
velocity, etc.). This legacy makes learning bach eas- Orlarey, and Letz 2012) the ability to operate with
ier for users accustomed to OpenMusic or PWGL. symbolic musical representation in real time; it

Agostini and Ghisi 23


Figure 11. A bach.roll Jean-François Laporte. The live.line objects used to
displayed in a patch used graphical parameters of display the custom-
in Carlo Barbagallo’s bach.roll are adjusted to positioned horizontal
Tatàmme, written for the display a nonstandard lines, no layering of other
Table de Babel, an score that is generated in Max graphical widgets has
instrument set conceived, real time during the been used.
built, and played by performance. Besides the

shares with note∼ (Resch 2013) and, to some operations such as constraint solving or rhythmic
extent, Antescofo (Cont 2008a) the goal of providing quantization. Its data representation capabilities
Max with advanced sequencing capabilities; it shares are confined to the standard Max list—which, in
with FTM (Schnell et al. 2005) and MuBu (Schnell turn, means that RTC-lib does not have a notion of
et al. 2009) the characteristic of implementing new score. On the other hand, all this makes RTC-lib ex-
data structures that are more powerful and complex tremely lightweight, and its composing modules are
than those natively provided by Max. It is a fact that easy to integrate individually into generic patches.
several aspects of all these systems, including bach, In contrast, bach modules are chiefly designed to
overlap. At the same time, each has its own peculiar interact with each other.
features that strongly distinguish it from the others.
For example, the quality and flexibility of musical
typesetting of MaxScore and InScore are clearly Current Status and Future Developments
higher than those of bach; the event-sequencing
capabilities are much more advanced in note∼ and The development of bach has been uninterrupted
Antescofo; the FTM object system is richer and more since 2010, and during the past few years the
complex; and the MuBu data containers are exactly library has earned a user base whose size we
tailored for the data types they are meant to hold. estimate to be around 1,000 people, according
The idea at the basis of bach is that all its modules to the download statistics and the interactions
form a coherent computer-assisted composition on the dedicated forum. The bach library can
environment, and its very goal is to make live be downloaded free of charge from the official
computer-assisted composition, in a responsive, project Web site (www.bachproject.net) and runs in
interactive world. conjunction with Max on both Windows and Mac
At least two other software projects share OS X.
this same perspective: Peter Elsea’s LObjects A large part of our development efforts go in
(peterelsea.com/lobjects.html) and Karlheinz Essl’s the direction of reliability and usability. Because
RTC-lib (www.essl.at/works/rtc.html), both predat- of the massive number of features of some mod-
ing bach by several years. The former is mostly ules, particularly bach.score and bach.roll, a
devoted to enhancing the list-processing capabilities custom, tag-based documentation system has been
of Max, with only a small number of objects dealing implemented, and it is being maintained to include
with problems more specifically related to MIDI. detailed descriptions and examples for every new
The latter implements both generic data processing feature of every module of the system. Furthermore,
and strictly compositional operations, with a lean- each object and abstraction is documented by a de-
ing towards post-serial and aleatoric techniques. It tailed help patcher and a complete reference sheet.
is a less powerful system than bach, as it does not Finally, a broad collection of tutorials covers most
provide graphical user interfaces, sequencing tools key concepts of the system and exemplifies some
(except for a simple step sequencer), or advanced typical usage scenarios.

24 Computer Music Journal


Figure 12. Aaron Einbond CataRT and bach. Grains associated with the grains such as control of
and Christopher Trapani played by CataRT are themselves. The content of synthesis or score
are currently developing a recorded in a bach.roll the bach.roll object can generation. (Image by
tool for controlling object as notes whose subsequently be played Aaron Einbond; see
corpus-based properties and metadata through CataRT, edited in Einbond et al. 2014)
concatenative synthesis represent the various place with a lambda loop,
through the interaction of parameters and descriptors or used for other processes

into bach.score and bach.roll, although we do


not consider bach as a tool for music engraving,
but rather for interactive manipulation of musical
materials.
Another major issue, already mentioned, is
the fact that the main strength of Max lies in
its interactivity and the quality of its graphical
interface, rather than ease of representing complex
algorithms or computational power. In fact, during
the implementation of the cage library, we ourselves
realized that processes that could have been easily
represented in a text-based programming language
required very complex patches to be implemented
through Max and bach. At the same time, as
discussed earlier, some specific operations may be
slower than optimal because of the mechanisms of
message passing in Max and llll passing between
bach objects, and because of the impossibility
of performing direct, low-level operations on list
nodes. The only currently viable solution to these
problems is implementing such operations as
external objects written in C. For this reason, we
provide an SDK allowing other people to code their
own externals while making use of the bach data
structures. Still, C coding for Max is intimidating
for most Max and bach users, and problematic
because subtle mistakes can cause serious stability
problems for the whole system. We are therefore
Medium- to long-term development plans include studying the possibility of implementing a high-
new major features to be implemented. One of the level, Lisp-inspired, textual language allowing users,
most important issues currently under study is the especially those familiar with Lisp, to write routines
absence of a page layout for score display: both for llll manipulation. This language will be JIT-
bach.roll and bach.score represent musical compiled on the fly and will therefore run at native
scores as a single, scrollable staff system, and the speed.
only way to achieve a sort of “page view” is to
use several bach.score objects, each showing one
portion of the same score. Although technically Examples
feasible, this approach is cumbersome and ineffi-
cient. A direct consequence of the absence of a page Figures 9 through 12 present concrete examples
representation is the fact that it is currently difficult of how bach can be used: a flexible sequencer
to print scores or generate PDFs directly from bach. for controlling a video-synchronized synthesis
One possible workaround for this is to export a system (see Figure 9), a hierarchically organized
bach score to a music engraving system, and to electronic score (see Figure 10), a graphical score
have that system manage page layout. As alluded obtained by refining the display parameters of a
to earlier, this process can be done automatically bach.roll object (see Figure 11), and a tool for
through LilyPond. Thus, we are currently research- controlling corpus-based concatenative synthesis
ing the possibility of implementing a notion of page (see Figure 12).

Agostini and Ghisi 25


Conclusions In P. Van Roy, ed. Multiparadigm Programming in
Mozart/Oz. Berlin: Springer, pp. 277–291.
After more than five years of development, the Anders, T., and E. R. Miranda. 2011. “Constraint Pro-
basic structure and working principles of bach gramming Systems for Modeling Music Theories and
Composition.” ACM Computing Surveys 43(4). Avail-
are rather stable. At the same time, it is hard to
able online at dx.doi.org/10.1145/1978802.1978809
foresee whether and when a version of the library (subscription required).
will be considered definitive. Many are the future Assayag, G., et al. 1999. “Computer Assisted Composition
challenges, including our wish to protect the total at IRCAM: From Patchwork to OpenMusic.” Computer
independence of the work, which has never received Music Journal 23(3):59–72.
any formal institutional support, and at the same Bonnet, A., and C. Rueda. 1998. “Situation: Un Langage
time to reach the largest possible user base and visuel basé sur les contraintes pour la composition
collect their suggestions and requests. musicale.” In M. Chemillier and F. Pachet, eds.
Recherches et applications en informatique musicale.
Paris: Hermes Science, pp. 65–74.
Carpentier, G., and J. Bresson. 2010. “Interacting with
Acknowledgments Symbol, Sound, and Feature Spaces in Orchidée, a
Computer-Aided Orchestration Environment.” Com-
The conception and the development of the bach puter Music Journal 34(1):10–27.
library have been partially undertaken while the Cont, A. 2008a. “ANTESCOFO: Anticipatory Synchro-
authors were students at the Course in Composition nization and Control of Interactive Parameters in
and Musical Informatics at IRCAM, and then as Computer Music.” In Proceedings of the International
composers-in-research at that institution. Computer Music Conference, pp. 33–40.
We wish to thank Carlos Agon, Arshia Cont, Éric Cont, A. 2008b. “Modeling Musical Anticipation: From
Daubresse, Emmanuel Jourdan, Serge Lemouton, the Time of Music to the Music of Time.” PhD
Jean Lochard, Mikhail Malt, and all the other people dissertation, University of California, San Diego.
at IRCAM who have supported and encouraged Didkovsky, N., and G. Hajdu. 2008. “MaxScore: Mu-
sic Notation in Max/MSP.” In Proceedings of the
the work, although it was not an internal IRCAM
International Computer Music Conference, pp. 483–
project. 486.
We also wish to thank Sara Adhitya, Jeremy Bern- Einbond, A., et al. 2014. “Fine-Tuned Control of Concate-
stein, Davide Bordogna, Joshua Kit Clayton, DaFact, native Synthesis with CataRT Using the bach Library
Phil Gaskill, Eric Grunin, Mika Kuuskankare, for Max.” In Proceedings of the International Computer
Andrew Pask, Andrea Sarto, and David Zicarelli. Music Conference, pp. 1037–1042.
Esling, P., and C. Agon. 2010. “Composition of Sound
Mixtures with Spectral Maquettes.” In Proceedings
References of the International Computer Music Conference,
pp. 550–553.
Agon, C. 1998. “OpenMusic: Un langage visuel pour la Fober, Y., Y. Orlarey, and S. Letz. 2012. “INScore: An
composition musicale assistée par ordinateur.” PhD Environment for the Design of Live Music Scores.” In
dissertation, Université Paris VI. Proceedings of the Linux Audio Conference, pp. 47–
Agon, C., et al. 1994. “Kant: A Critique of Pure Quantifi- 54.
cation.” In Proceedings of the International Computer Laurson, M. 1993. “PWConstraints.” In Atti del X
Music Conference, pp. 52–59. Colloquio d’Informatica Musicale, pp. 332–335.
Agostini, A., E. Daubresse, and D. Ghisi. 2014. “cage: Laurson, M., and J. Duthen. 1989. “Patchwork, a Graphical
A High-Level Library for Real-Time Computer-Aided Language in Preform.” In Proceedings of the Interna-
Composition.” In Proceedings of the International tional Computer Music Conference, pp. 172–175.
Computer Music Conference, pp. 308–313. Laurson, M., and M. Kuuskankare. 2002. “PWGL: A
Anders, T., C. Anagnostopoulou, and M. Alcorn. 2005. Novel Visual Language Based on Common Lisp, CLOS
“Strasheela: Design and Usage of a Music Composition and OpenGL.” In Proceedings of the International
Environment Based on the Oz Programming Model.” Computer Music Conference, pp. 142–145.

26 Computer Music Journal


Lemouton, S. 2011. “Using Gecode to Solve Musical Schnell, N., et al. 2005. “FTM: Complex Data Structures
Constraint Problems.” In G. Assayag and C. Truchet, for Max.” In Proceedings of the International Computer
eds. Constraint Programming in Music. Hoboken, New Music Conference, pp. 423–426.
Jersey: Wiley, pp. 103–132. Schnell, N., et al. 2009. “MuBu and Friends: Assembling
Puckette, M. 2004. “A Divide Between ‘Compositional’ Tools for Content Based Real-Time Interactive Au-
and ‘Performative’ Aspects of Pd.” In Proceedings of the dio Processing in Max/MSP.” In Proceedings of the
First International Pd Convention. Available online at International Computer Music Conference, pp. 423–
puredata.info/community/conventions/convention04 426.
/lectures/tk-puckette/puckette-pd04.pdf/at download Taube, H. 1991. “Common Music: A Music Composition
/file. Accessed January 2015. Language in Common Lisp and CLOS.” Computer
Resch, T. 2013. “note˜ for Max: An Extension for Max/MSP Music Journal 15(2):21–32.
for Media Arts and Music.” In Proceedings of the Truchet, C., and P. Codognet. 2004. “Solving Musical
International Conference on New Interfaces for Musical Constraints with Adaptive Search.” Soft Computing
Expression, pp. 210–212. 8(9):633–640.

Agostini and Ghisi 27


Copyright of Computer Music Journal is the property of MIT Press and its content may not be
copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for
individual use.

You might also like