High level languages for Computer Music

Alexander K. Ames sasha@cse.ucsc.edu Univ. of California, Santa Cruz

Abstract
Programming languages have been used for many years for making computer music. Given the complexities of low-level music programming languages, there have been attempts to implement high-level ones that may better t music composers needs for expressing musical ideas.

1.

Introduction

The application of computer systems to music has ushered many new possibilities for the artform not before realizable by traditional means. Sounds that before were only producible through direct interaction of a human with an instrument or through her voice could now be generated by machines. Since the onset of this musical there has been a corresponding need for a systematic means by which the sound shall be organized into music. Additionally the earliest machines were not capable of running the complex user interfaces we may nd in the computer music production systems of today. Thus, programming languages for computer music were born [6]. Much like general-purpose computer languages, computer music programming languages may be categorized as low or high-level languages. Low-level languages provide a much ner level of granularity into the workings of the generated sound but will result in copious amounts of code for musical representations. On the other hand, high-level languages for music can provide more general models for musical expressiveness, but may lack the total control of the sound possible with lower level ones. These languages shall need to differ from other programming languages (though we may see they may not so much) in that they have a very particular user base. Music composers arent necessarily computer scientists. They generally are trained in conventional music notation, theory and composition techniques. Thus, they shall most likely nd languages that best t that training as ideal to use. Some may be well inclined to the low-level sonic representations of their music, but others may nd they rather have one that they are more accustomed to. The high-level language designers must consider this in their attempts to provide a new formal means for musical expression.

2.

Low-Level languages

Before computer music systems had fancy software applications for synthesis and composition, and before the hardware could even support such applications, low-level programming languages served as the primary interface for those who wished to make computer music. I consider certain computer music languages to be low level due to somewhat analogous characteristics they may have with low-level general purpose programming languages. They function as to give the composer/programmer ne grain control over the synthesis of individual sound elements

much like instructions in an assembly language shall give the nest grain of control over the processing within the computer system. The Music-N 1 family of languages would stand for much time as general paradigm for the low-level computer music language. The rst of these was Music I, developed by Max Matthews in 1957 at Bell Laboratories in New Jersey. This version ran on the IBM 704 computer with the rst ever 12-bit digital-to-sound converter to actually generate the music. Max and other would continue to develop subsequent variations of the language. [8]. Music III would contain a concept in the language that would be stand as the central feature for all future language avors in the family: the unit generator. Unit generators are simple instructions for producing or modifying sound with various parameters. Examples of such are oscillators, lters and ampliers. For example, an oscillator may take as parameters of duration, frequency and amplitude. Thus, for someone to program in a Music-N language, she is combining many unit generation instructions to generate the desired sound or musical composition as output. Some of the much later variants in the Music-N family introduced some high-level programming language constructs to allow for advances in composition techniques. One such language, Cmix, kept many of the unit generation concepts, but added some c-style syntax with its min-C2 interpreter with certain common language constructs such as variables, if-then statements, and loop control with for and while loops. These constructs would allow the language to used for the writing of programs to generate algorithmic compositions [7]. Some computer music composers may nd they like having the ne-grain control of sound possible with these languages as they are able to shape ever such nuance within a piece of music. However, there are some big disadvantages of composing music using such low-level languages. Composing computer music can be much like programming, in that it involves many iterations of modication and composition debugging. Thus, using these languages at their ne-grained level of music representation shall yield very large numbers of instructions and the modication and debugging processes can become very tedious. Given that those who would be primarily interested in creating music are composers, and composers may not necessarily be skilled programmers, they desire languages that shall be more easy for them to relate to.

3.

High-level language examples

It should be understood that most composers with formal musical training consider the concept of a note as a fundamental unit. This is a somewhat of an abstract concept in contrast to the more fundamental sounds created by unit generators in the Music N low-level languages. Moreover, composers consider certain properties or dimensions of the notes, those being pitch, time/duration, volume (also amplitude or velocity) and tone color or timbre. These basic constructs allow for the generation of more complex musical constructs, such as combinations of pitches called chords, variations in time called rhythms and tempo changes, dynamics: changes in volume, and articulation which may contain time and volume changes. The ability to express these ideas is very important to the composers of music. As a consequence, there have been quite a few attempts to provide composers with higher-level programming languages with which to compose music. In order to provide support for musical constructs with various levels of abstraction, the languages too must enable the representations of those constructs with relative ease to the composer.

3.1. Formes
The Formes language, developed by Cointe and Roted,[2] was perhaps a rst attempt at providing an objectoriented language for the composition of computer music. In devising their language concept, the authors looked
1N 2 stands

being some number, but we shall see related languages not using the naming convention for minC-is-not-C, basically a subset of C

at existing object-oriented systems, such as Smalltalk. Their interest in adopting object-oriented principles in their design attempts stems from what they see that these principles best match the requirements of what is necessary for music composition and synthesis. However, they did see the lack of the time-component as a built-in language construct that would be necessary for musical composition. Formes is modelled after the general-purpose functional language of Lisp. Moreover, it was specically made possible by some attempts to add object-oriented features to the VLisp or Le Lisp dialects of Lisp. Within this framework, each Formes object is also a Lisp function. In Formes, the types of objects a programmer may dene falls under a few categories, but are primarily hierarchical. One may dene process objects to perform different aspects of a composition. A hierarchy exists of theses processes in that they must be created from a top-level or God process, and processes may have sub-processes. The organization of the process hierarchy comprises what Formes authors refer to as the genealogical tree. To handle the time component necessary to produce music, Formes allows the programmer to dene monitors. Monitors, are like Hoares monitors. In Formes they function as the scheduler of a process and sub-processes. The act of scheduling the processes by the monitors generates the calculation tree, which provides a total order of method invocation from various process objects [3]. The authors give an example of a simple Formes process hierarchy by breaking down a simple note denition. One possible denition of a note contains attack, sustain and decay components. Thus those three are the subprocesses of the note process. Individual note instances can be differentiated in terms of dynamics (volume) by dening different types of note instance object processes, such pianissimo, mezzoforte, and fortissimo. A superprocess may then comprise the specic notes. When the calculation tree is created for the new super-process, it breaks down into the individual notes, and then the individual note components. Another example shows Formes handle pattern matching of series of pitches to schedule the performance of the pitches. This allows for note sequencing in Formes and can be done for multiple simultaneous voices, a necessity for any musical composition system. A nal noteworthy feature of Formes is their approach of using function application to implement message passing. Message passing is necessary so various processes can interact to shape the musical output. Based on Lisps procedure for evaluation of expressions, one may specify a recipient object name, type of message and message parameters.

3.2. Formula
The Formula language for programming computer music also attempts to address the lack of time-related features in general purpose programming languages. It is based upon a general purpose programming language called Forth, but with a variety of added functions and control structures. Formula was designed with score interpretation, algorithmic composition, and interactive music in mind. The latter is made possible by the presence of a command interpreter in the language that allows for some realization of some musical ideas in real-time [1]. Formula takes an approach very distant from the Music N languages. It does not have anything to do with the synthesis of sound. Instead, it directs the musical output in different synthesizer or other sound producing modules via a MIDI interface. Thus, the output of the language may lack the ultimate expressiveness that one may be able to obtain from using a language that works on unit generators, producing direct synthesis of the composition; the limitations upon expressiveness shall come though those imposed by the synthesizers and the MIDI language. Similar to Formes, Formula also has the notion of a process. However, the way the processes function differs. Processes are more like functions than objects, and these functions may be applied to each other. For instance, one may dene a process to dene the specic pitches which in turn can be applied to a process to dene the rhythm to be taken on by the pitches, called a timing sequence generator. Formula handles time by having processes arranged sequentially or in parallel, so there is no scheduler. The time component of an individual process is based on how long it should last over a specied period of time, 3

specied in an arbitrary unit such as beats or measures. Thus, each process shall have its own virtual time system. The mapping of these times to absolute time (measured in seconds) comes from other processes mentioned in the following paragraph. Additionally, to control the various expressive aspects of the music, Formula has the ability to dene auxiliary processes. These processes differ from the others as they must be applied to others; they are not fundamental within the composition. They allow for tempo control which will map the note rhythms to absolute time, articulation, and volume control. These functions may be either applied to each other or concatenated to provide varying behaviors over some period of time. While the Formula language may enable some musical expressiveness through the ability to compose processes to handle the various aspects of the music, it has some noticeable weaknesses. First, The Forth code syntax is not the easiest to understand. Second, it is completely untyped, which may give rise to programming errors. It was the authors intent to reimplement Formula with c++. I have not seen in the literature whether they were successful or not. While a c++-like language for musical expression may enable some more powerful constructs, through objectoriented features such as inheritance and data abstraction, I would guess it would would not simplify anything for the composer. The composer would now have to understand those object oriented concepts in order to have processes applied to each other as many interfaces and instantiating subclasses would have to be dened. This would be a move away from the ease of functional programming possible in former described version of Formula.

3.3. Haskore
Paul Hudak et. al developed a programming language for computer music based on the Haskell functional programming language. This variant, called Haskore [5], has one primary motivation of working towards a language in which composers may express musical ideas that might be too cumbersome by conventional musical notation. For example, (as provided by the authors) a composer may want to express seven notes over the span of 5 beats, or even better use non-integer divisions of time. This is possible to do at a low or synthesis level, as sound must be expressed as absolutes which can be calculated, but not at a higher-level of expression. Moreover, the authors were looking for an almost algebraic approach to expression of music, which in turn could be used for its realization. Part of the foundation for Haskore came from a study of the complexity of modelling musical performance using the Haskell language as a tool for formal representations [4]. The basic gist of this model involves that there exists in the process of making music a relationship of the player and her instrument. Specically, the player must, in a sense, read from the instrument while she uses it to make one sound following the next, and that will shape the outcome of the music. This can extend to an ensemble where the individual performances of the musicians in the ensemble inuence each other. In addition, the musicians are following a predened score. Thus, we have models from which the score becomes interpreted to form a concrete performance. Haskore builds data types with which to express music. The basics are notes, which consist of pitches and durations and rests, which only have durations. Haskore denes the Music data type as a recursive data type, in which it can be line of notes, a tempo scale some Music, a transposition of some Music, Music played in series, or Music played in parallel. Given these denitions and properties, various functions can be created to provide transformations to the Music type. Haskores design has the interesting aspect of having the score and performance or interpretation as separate models. Some abstract data object of type Music becomes the score, and various functions can be dened to interpret it to produce a performance. This performance may have many varieties of nuances that one may nd in the interpretation of music, such as tempo, articulation, or volume changes. For example, one could write a function or series of functions in Haskore to create a unique performer. This performer would always play the last note of every four measures slightly at. Or, perhaps adding slight randomness to each note in pitch and duration to give a more imperfect or human feel to a performance would be possible with a performance function. 4

Haskore exists as a language without any predened synthesis technique as inherent to the language itself. The authors have written a translator for the language that generates CSound language output. CSound is another low-level unit generator synthesis language in the Music N family. One nice advantage to doing it this way is that it may allow a composer the possibility of making necessary modications to the low-level musical output specication without having to do the bulk of the composition in that language. This is analogous to a programmer modifying binary machine code after it has been compiled from a already written program in the C language.

4.

Comparison

While these three languages take different approaches to solving a general problem. They all seem to have some concepts in common. One common concept is that all the languages contain mechanisms by which some dened musical idea may be transformed. This is perhaps the most complicated within the Formes language, as a separate process must be enabled to handle the transformation of the music. The process must be scheduled with a monitor to be syncronized with the music. This is simpler in Formula, which has the built-in auxiliary processes that can be combined with each other and applied to a process generating musical notes. Finally, Haskore allows the denitions of functions to transform music to the composers liking. Another feature handled by all, but in somewhat more different manners, is that of introducing a time component to a programming language, where generally commands are executed in a specieed order as soon as possible. Formes has monitors to schedule processes into some time system. The monitors themselves are derived from the God process scheduler, which has ticks at some specied granularity (by default: 100Hz). For Formula, within each process there is some amount of virtual time specied. The act of applying a time deformation process to that shall give it life in the sense of creating the absolute time component we nd in any interpretation of written music. Haskore has time represented for each note (or rest) primitive in a virtual sense in terms of beats. Total virtual time is the total of all sequenced duration values. That, in turn, shall be mapped to real-time through a performance function. Where the languages differ greatly is in their syntax. This is, of course, due to each being based on different general purpose programming languages, but that too has further consequences for the composer. One of Formes biggest weaknesses that I see is that it is based on an attempt to add object-oriented features to a functional language. While possible both in theory through formal representations and in practice through implementation, it may not be best suited for the intended user base, as it results in a far from elegant object representation. Formulas syntax is not so easy to understand at rst glance, but the semantics of the language as applied to music is much more straightforward and elegant. Finally, Haskore has perhaps the most elegant representation for music and appears terric for representing many musical concepts, its quite possible that some other concepts (regarding issues of timbre) in particular and perhaps also articulation, where it may be lacking.

5.

Conclusion

High-level programming languages for computer music present an interesting alternative for composers who wish to take advantage of the technology or create new kinds of music. However, it seems that these particular examples might not have exactly caught on with the composition community despite some nice features for musical creation and some elegant language design features. The fact of the matter is that music composers often are concerned with the nature of the sound itself, so may embrace the use of the low-level language as part of their craft. The other part is the advent of ever improving applications for composition that can be used with sound output from a low-level language, if not completely supplant its need. These graphical tools have often become just as powerful for creating musical expression as what may be possible through one of the high-level languages, and much easier to learn. Thus, for most composers the choice has become obvious.

Nonetheless there may be composers of computer music who wish to express music in this fashion. Thus, future language designers may have some prior reference points to look in terms of what may be possible or what they could do differently. Ultimately, the composers shall judge if the language design effort is fullling.

References
[1] D. P. Anderson and R. Kuivila. Formula: A programming language for expressive computer music. Computer, 24(7):12 21, 1991. [2] P. Cointe and X. Rodet. Formes: An object and time oriented system for music composition and synthesis. In Proceedings of the 1984 ACM Symposium on LISP and functional programming, pages 8595. ACM Press, 1984. [3] P. Cointe and X. Rodet. Formes: Composition and scheduling of processes. In C. Roads, editor, The Music Machine, pages 405423. The MIT Press, 1984. [4] P. Hudak and J. Berger. A model of performance, interaction, and improvisation. In Proceedings of International Computer Music Conference, pages 18. Intl Computer Music Association, 1995. [5] P. Hudak, T. Makucevich, S. Gadde, and B. Whong. Haskore music notation - an algebra of music. Journal of Functional Programming, 6(3):465483, 1996. [6] G. Loy and C. Abbott. Programming languages for computer music synthesis, performance, and composition. ACM Comput. Surv., 17(2):235265, 1985. [7] C. Roads. Music languages. In C. Roads, editor, The Computer Music Tutorial, pages 785818. The MIT Press, 1996. [8] C. Roads and J. Strawn. Introduction to digital sound synthesis. In C. Roads, editor, The Computer Music Tutorial, pages 405423. The MIT Press, 1996.