THE CREATION OF MUSICAL SOUNDS FOR

PLAYBACK THROUGH LOUDSPEAKERS

BY DAVID MOULTON

Chairman of the Department of Music Production and Engineering

Berklee College of Music
Boston, MA.

The evolution of multi,rack recording of popular music has led to significant changes in style and
aesthetics, compared to live performances and the recordings derived from them. Central to these
issues is the idea that music may be represented by a single waveform over time and the emergence
of a "loudspeaker music" aesthetic based on the tension that exists between the A+B and A-B
elements in stereophonic playback of multitrack recordings. This paper reviews the technology
systems and usages that have influenced these changes, and discusses the validity of the current
assumptions and practices in terms of psychoacoustical principles. Particularemphasis is paid to
the idea of stereophony as it is employed in multitrack popular music recording.

1. HISTORICAL BACKGROUND creative accomplishment that has gone on in the recording

Sound recording has been viewed since its inception as a
method for the storage and reproduction of acoustical musical
events. This archival activity has obscured an emerging and
equally important facet of sound recording: its use as a prima-
ry medium for the creation of music. This creative use f'mds
many of ils origins in popular music, the evolution of popular
electric musical instruments, and the mulfitrack efforts of Les
Paul and others,
While stereophony has evolved from efforts to enhance
the listening experience (particularly of classical music),
multitrack recording has evolved from efforts to enhance
popular music performance, by artificially expanding often
limited resources and by deferring musical decisions until
the moment of fixing the master recording. The practition-
ers of multitrack recording have been highly pragmatic in
their approach and have sought, in common-sense musical
and technical ways, to improve the apparent quality of their
performances and recordings. The tremendous commercial
success their work has enjoyed has in turn led manufactur-
ers to eagerly support theft multitrack endeavors and re-
spond quickly to their expressed needs. A recording indus-
try based on this synergy has evolved extremely rapidly. At
the same time, comparatively little has been done to assess
the underlying principles and aesthetics of this method of
creating music, or the psychoacoustical implications and
validity of those principles. The evolution of multitrack
recording as a primary music-creation medium has been so
rapid that there simply hasn't been time yet to evaluate and
place into historical perspective the immense amount of
of popular music since 1950.
Suffice it to say that multitrack recording has proceeded on
the premise that music consists of a multiplicity of voices,
each of which can be represented by a single electrical wave-
form, and that successful, beautiful, and exciting music can be
created from a set of such waveforms. It should be further
noted that the preferred performance of such music is by loud-
speaker pair. Our efforts have been mainly directed at the de-
velopment of a craft and technology to allow us to create,
modify, and mix the waveforms of such a set. Little attention
has been paid to the exploration of other assumptions about
and possibilities for the creation of music for loudspeakers.
2. TECHNICAL PREDISPOSITIONS OF
MULTITRACK RECORDING PRACTICE
In many respects,art is shaped and driven by the technology
which surrounds it. The three-minute song, for instance, is
clearly a response to the physical constraints of the Edison
cylinder and its descendents, and the applications and prac-
rices of stereophony are equally rehted to the physical and
psychological forces inherent to the mulitrack recording pro-
cess and its environs. The assumptions underlying the contig-
urafion and operation of the modem recording studio have
profoundly affected the nature of the music created in it (such
as the evolution of the A+B/A-B aesthetic process noted be-
low). Therefore it is important to be aware of the configura-
tion and practices of the multitrack recording process, as well
as those of the end-user audience, in order to more fully an-
derstand the music that is created with it, as well as the appro-

AESafl3INTERNATIONAL
CONFERENCE 161
MOULTON
priate design goals for the hardware audio systems used to
transmitthatmusic,
2A. Microphone and Monitor Usage
Multilrackrecording is based on the assumptionof multiple,
separateand independent voices. This has led to a predisposi-
lion for isolation of insmunental sounds. A primary method
for obtaining isolation is through the placement of micro-
phones in the near-field of a musical instrument. Typical
placements are usually within inches of the source,with con-
siderableattention being paid to the exact placementas a pti-
mary determinant of timbre. Further isolation is obtained
through the preferred use of microphones with cardioid and
hypereardioid polar patterns. These placement traditions are
sufficiently pervasive that even when near-ideai isolation is
obtained via overdubbing, near-field placements continue to
be preferred, except for special effect. The convention that
precise microphone placement in the near field is a primary
determinant of timbre is carried even further as a matter of
course with the mixing of multiple near-field microphone
placements for the same instrument to obtain thedesired tim-
bre. Sometimes, electrical pickups will also be mixed with
near-fieldmicrophones,
At the playback end of the recording/reproductionprocess,
the evolution of monitor loudspeakersand conlrol room con-
figurationshave arisen from a variety of designneeds, histori-
cai traditions and acoustical philosophies. The goal of much
of this designeffort and usage has been to eliminatethe effect
of the controlroom on the audio signal, based on the assump-
tion that the listening room will add coloration to and will
mask the true musical character of that audio signal.l,2 A
cenlral ingredient in such design methods has been the at-
tempt to minimize the amount and effect of controlroom re-
flections on the listener through the use of highly absorbent
rooms, geometriesthatprevent early reflectionsfromreaching
the listener,near-field monitor placement, etc. A result of this
approach has been the creation of rooms that Supportabner-
really strong and stable phantom images (which permit users
may derive maximum audibility fromthe effect of amplitude-
distribution punpots). At the same time these rooms yield
comparativelyaudible interferenceeffects (i.e. comb-filtering)
around the median plane. Detailsof reverberanceand time de-
lay are also extremely well revealed. Dynamic range may ap-
preach the limits of human thresholds,
Such roomsbear littleresemblance to the end-userenviron-
ments for which the recordings are intended. It is now ax-
iomatic that music mixes are subsequently evaluated in a
range of such end-user environments and then remixed to
compensate for undesirable qualities perceived in those end-
user environments. Producers and engineers also employ a
commonpractice of using their own small near-fieldmonitors
in production control rooms to minimize errors of this sort
and thereby shortenthis poslproductionprocess,
2B. Signal Processing
The recording console carries multiple parallel channels,each
designedto condition and process an audio signal.Apart from
the many functionsdesigned to facilitate production,consoles
have as a primary function the interconnectionof these chart-
162
nels via an arrayof mix buses. Some of the buses areused to
collect signals for global signal-processingeffects (such as re-
verb) which are then routed throughyet otherchannels.These
parallelchannelsare thensummedto two buses(for stereo)
for final mixing, distributedto theses buses via amplitude-dis-
tribution panpots. Monauralversions of mixes are obtained
via a simple summationof the stereo buses, usually for moni-
toting purposes only, and then again later in broadcast tram-
missionfor radioand television.
Each channel will have equalization (the modification of
frequencyresponse of an audio signal)available.Equalization
is, along with microphone placement,consideredto be thepti-
mary determinantof timbre. In contemporaryconsoles,equal-
ization is fairly flexible,with usually four separate circuitsof
so-called"parametric" control (i.e. control of amplitude, fre-
quency and bandwidth of the response modification) per
channel. The stereo localizationof an audio channel is deter-
mined by a panpot (usually dual potentiometerswith oppos-
ing taperson a common shaft). This circuit is used todayas
the primary determinantof stereo, and the practicingassump-
tion is that relative distribution of amplitude to the Left or
Right bus determines the location of the perceived stereo-
phonicphantomimagefor thatparticularaudiosignal.(Man-
ufacturers also view stereo itself in this light, ff a device has
two outputs and the amplitude of a signal distributedto those
two outputs is controlledby a panpot, the deviceis by defmi-
tion a "stereo" device.)
Gain-manipulationdevices (compressors,expanders et al.)
are not generally fitted to all channels of a console,but are
available widely in most studios(typicallya minimum of one
for every two channels of audio on the console). These de-
vices were not originally developedto enhance the musicality
of a signal, but to avoid overload distortion and system noise
while recording acoustic sounds whose dynamic range ex-
ceeded that of the audio system. Recording engineers, musi-
cians and producers have noticed the substantial impact of
such devices on the timbre and musical quality of a sound,
and have increasingly used companders for the purpose of
manipulating musical values even as the available dynamic
range of the audio channel itself has increased to a point
where overload distortion and/or objectionable system noise
are no longer seriousconstraintsin the recordingprocess.
Signal processing in the time domain has proved to be one
of the most fruitful and musically compelling realms of craft
in multitrackproduction, for reasons relatedto the psycholog-
ical nature of the echo-locationportion of our hearingmecha-
nism. This realm includes thegenerationand reiterationof de-
lays ranging from approximately 100 microseconds to about a
second.Due to the comparativelyhigh cost of delay lines and
the forbiddingly broad and complex range of possibleconfig-
urationsthat might be desired,delay lines have not becomean
integral part of each channel, and reverberators remain "out-
board" units, usually dedicated to one functionat a time (al-
though often time-shared through overdubbing, in a multi-
track cost-saving tradition).
One of the earliest and still most powerfulmusicalgestures
to emerge from recording is the reiterated"tape echo," audio
signal delays of greater than 50 milliseconds(and usuallyless
than 500) generated originallyby theplayback head of a spare
AES 8th INTERNATIONAL CONFERENCE
 THECREATIONOF MUSICALSOUNDSFORPLAYBACK
deck, and sometimes looped for multiple iterations. This
became a cliched vocal reverb effect in the 1950s and has
proved to be tremendouslyuseful for a range of musical ges-
turesover the interveningyears,
Shortdelays (ca. 5-50 milliseconds)have proved to be very
usefulfor a broad range of uses, such as the simulationof in-
strument doubling and fimbral (spectral) modification of a
signal (due to comb filtering). The mixing of a signal with its
delayed iteration generates such comb f'fiteringand subse-
quent timbral interest. This can also take the form of doubling
or flangingif the lime base of the delay is modulated. In addi-
lion, the distribution of short delays to the other channel of a
stereopair results in much stronger spatiallocalization(due to
the precedence effect) than can be obtained by amplitude pan-
ning. Stereoflanging, doubling,chorusing and a host of other
effectshave alsobecomecommon.
Live doubling(the overdubbing in unison of the same per-
former) is a remarkably beautiful sound, unique to multilrack
_c_rding. Whetherdone in stereo or mono, it adds substantial
intensity,impactand interest to the part. This sort of doubling,
liketape echo, hasbecome a durable cliche. Interestingly,uni-
siondoublingsof twoidentical instrumentsare discouragedin
traditionalorchestration,particularlywhen the instrumentsare
rich in upper spectra.31speculatethat the interferenceeffects
which prove to be of interest in the multitrackrealm are less
· so in the traditionalorchestral realm due to the comparative
richnessof reverberantinformationin thatrealm,
Withthe advent of digital reverberators,artificialreverbera-
tion has becomea powerfuland extremely flexibletool.Using
a broad army of algorithms for modeling the reverberant be-
haviorof rooms,reverberatorshave led us to remarkablynat-
ural-sounding artificial reverberance. In addition, numerous
non-naturalsoundsand special effects, such as so-called"gat-
ed reverb" (the sudden truncationof an extremely reverberant
sound),have evolvedinto normal productioneffects,
2C. MUSICSynthesis
Music synthesis has been with us since the advent of the
baroquepipe organ.After numerous developmentsin synthe-
sis duringthe firstpart of thiscentury, the analog voltage-con-
trolled synthesizer developed by Moog and others in the
1960sbecame the first widely available synthes r. In keep-
ing with multitrack sensibilities, the voltage-controlled syn-
thesizer treats the musical voice as an electrical waveform,
Other synthesismethods have followed in this tradition,
The most significant, enduring and original aspect of
Moog's design is his development of an integrated control
system,which defmes musical parameters in eleca'ical terms
(i.e. one octave = 1 volt, etc.), and applies this simple, focused
system to the physical elements of frequency (corresponding
to pitch), amplitude(loudness)and spectra (timbre).The other
key elementin Moog's control system is the envelopegenera-
tor, which creates control time-contours for each individual
sound, permitting extremely flexible shaping (if not varying
expressive nuance) of the loudness and spectral character of
individualnotes. Sequencing is an extension of synthesis con-
trol languageto permit theglobal controlof multiplesynthesis
parts in real- or step-time, as well as the creation of more ex-
pressive note-to-note progressions. The simultaneous emer-
AES8th INTERNATIONAL
CONFERENCE
THROUGHLOUDSPEAKERS
gence of the Musical InstrumentDigital Interface(MIDI) lan-
guage, personal computers and inexpensive multitrack
recorders has fueled explosively rapid growth of music ere-
ation utilizingthis sort ofcontrol system.
The full implicationsof this group of developments in syn-
thesis, particularlysequencing, cannot yet be assessed, but it
may prove as signficant to the history of music as the deveb
opment of the chromatic keyboard. As the technology ma-
tures, we can expect to see the reintegration of the acts of
composition and performance at a high level of sophisfiea-
tion in both areas, satisfying both the creative organizational
urge of the composer and at the same time the need for ex-
pressive nuance of the peformer, resolving a dichotomy that
has existed in Westem Music since the development of nota-
tion, a dichotomy that has on the one hand led to some of
Westem Music's greatest achievements (polyphony, the or-
chestra, common-practice tonality and modulation), but that
has also proved frustrating and limiting for performers and
composersalike, for a variety of practical, economic and crc-
ativereasons.
A varietyof methods of synthesishave evolved since 1960.
In general, the origins of synthesis are firmly rooted in the
work of Fourier and Helmholtz, in the assumptionsof wave-
form constructionand resonant structures as fixed elements.
These methods include:
- Additive synthesis: synthesis by creation of each sinu-
soidal partial of a sound, building timbral character through
the independentgenerationof each parfial's frequency,ampli-
tudeand (theoretically)relative phase.
- Subtractive synthesis: synthesis which begins with com-
plex waveforms and removes spectral content by filtering to
achievedesired fimbralcharacter.
- FM synthesis: an array of algorithms in the digital realm
that permits the creation of any spectralconfigurationthrough
thegenerationof sidebandsby the modulationof frequency of
one sinusoidal wave by another, where the amplitude, fre-
quencyand phaseof both carrier and modulator,and the
amount of modulationareall preciselycontrolledover lime.
- Sampling: the recording of single notes of instrumental
sound and treating the waveforms of those notes as musical
voices,controllingthem with the same control languageused
for other synthesis procedures. This modality has many ad-
vantages in terms of mimicking traditional instruments, but
also has proven to be fraught with difficultiesrelatedto (a) the
range of expressivenuance of acoustical instruments and Co)
problemspertaining to the audibilityof frequency shiftof the
formantof an acoustical sound. A special case of sampling is
the so-called "drum machine," which combines samples of
drum kit sounds with a small sequencer which permits the
generation of complex(?) patterns of drum sounds over time.
- Resynthesis:a techniquewherein sampledsounds are aha-
lyzed and converted into an equivalent additive, subtractiveor
other synthesis moclality so that they can be manipulated
within the frame of reference of that modality.
- LA (Linear Arithmetic) synthesis: a hybrid morality com-
bining sampled sounds with subtracfivesynthesis. Typically,
the beginning or attack of the sound is a recorded sample and
this attack is spliced to a sustaining waveform generated by
subtractivetechniques.
163
MOULTON
Other synthesis techniques,not necessarily dependentupon
the ideas of Fourier and Helmholtz, have emergedbut not en-
tered into general usage. These include modalities such as
VOSIM and granular synthesis,which are based on principles
generally outside of the conceptualrealm of fixed note stmc-
tures derived from waveformsAExperimental modalitiescon-
tinuetobe developed,
3. STEREOPHONY
Stereophony has become such a fundamental and ubiquitous
element in popular music production that someof its key fea-
tures need to be discussed briefly. The original notion of
stereophony lies in the idea that if a reverberantspace is ob-
served from two seI)aratebut nearbypoints simultaneously(as
our ears do), we can derive from the resulting signalsan illu-
sion of that space and the position of sound sources within it.
From this idea we derive the conventional two-
microphone/two-lou_er recording/playbackstereophon-
ic system. Technically, stereophonic audio is a dual-channel
transmission medium based on the premise that spatial infor-
marion that is both entertainingand satisfyingcan be included
as a function of amplitudedifferences(A-Amplitude)and time
differences (A-Tune)between the two channels of transmis-
sion. In multitrackrecording we mostlyuse an army of mono-
phonic signals that are mixed and dislributed with amplitude
differences (via pan-pots) to the two stereo channels. The
original sources are mostly single monaural signals (either
acoustically recorded in a space or electrically generated)that
are electrically summed with varying distributionsof ampli-
tude between the two stereo channels for each monaural
source signal. In keeping with the above characterizations,it
is possible to approximately define in the abstractthe physical
parameters of the stereophonic signal as follows:
the stereophonicartifact consistsof a pair of signalsderived
either from a common and simultaneously recorded acoustic
sound source or from a common electrical audio signal
source, so that the pair have a fixed or locked phase relation-
ship. The pair's common elements have less than 10 decibels
amplitude difference between them and less then 50 millisec-
ondstimeoffset.5
The phantom image, an illusion derived from the (A+B)
component of the stereo signal, is a clearly perceived image
localized(usually)betweentwo loudspeakers. It is theresult
of a peculiarity of the stereophonicplayback system vis-a-vis
our auditory localization process. In pre-technological nature,
no two sources are phase-locked, and our echo-location sys-
tem relies upon this fact: we localize through the Precedence
Effect, consciously identifying direction of the fa'st arrival of
a phase-locked series of artifacts as thedirection in which the
source of that sound lies. All subsequent phase-locked arti-
facts are, in pre-technologieal nature, reflections.6When two
loudspeakers each emit the same sound artifact, a listener
equidistant from those loudspeakers perceives two phase-
locked artifacts arriving from different directions simultane-
ously. This is paradoxical, because it represents a condition
that 1) does not occur in pre-technological nature, and 2) our
neurological perception system is not equipped to identify,
The resulting perception is the illusion of a single sound
source located somewhere along the median plane between
164
the actual sound sources.7 It has been generally assumed in
the recording industry that the phantom image is a common-
sense adjustment, a neurological intuition that the actual
sourcemust be somewherebetween the two actual artifacts.I
speculate that the auditory system in fact infers a more com-
plex situation, to wit: that those identical phase-locked,
equal-time-of-arrival, different-angleartifactsarein factper-
ceived as the first among many reflection artifacts of a sound
whose direct artifact, for some reason, was not perceived,and
that the locationof that unperceived imaginary sourceis to be
inferred from all of the reflected artifacts. In the case of a
loudspeakersystem whose two sources are in polarity,the sit-
nation is comparatively simple: the actual artifacts suggest
reflected paths from imaginaryside walls equidistant from an
imaginary source located somewhere (depth, for convention-
al loudspeakers, is ambiguous) along the median plane.
When the two sources are not in polarity, the situation is
more confusing, because the reversal in polarity yields a pair
of artifacts that suggest a greater paradox: the imaginary and
not-perceived source must be someplace other than the re-
gion where the sound is coming from; in fact it must be
someplace other than any normal place in the space in which
we are listening.
In popular music multitrackrecording practice, the mono-
phonic (A+B) element is played off against both (A,B) and
(A-B) elements of stereophony, as will be described below.
The clearly audible differentiation between (A+B) and the
other elements is used to musical and dramatic effect. The
monophonic origins of these sounds are usually quite audible
(particularly in a reflection-controlled control mom or ane-
choic listening environment). Often, the sound of an acousti-
cai stereophonicrecording (usuallyderived from a coincident
or an AB microphone placement on the dram kit) serves as a
sort of sonic "glue" to hold the stereophonic soundstage to-
gether, while (A-B) and (A,B) components spread the re-
mainder of the elements on the soundstage as widely as pos-
sible for maximum stereophonic impact. This process has as
its counterpart in acoustical stereophonic recording the ad-
justment of amplitude of the "side" information in a middle-
siderecording.
4. THE EVOLUTION OF "RECORDED MUSIC
AESTHETICS" IN POPULAR MUSIC
The effectiveness of the illusion of acoustical sound
reproductionis sufficientlycompelling that it has causedus to
mostly overlook the physical reality behind that illusion.
When we listen to sound recordingswe rarely notice theplay-
back loudspeaker per se; instead we hear pianos, singers,
voices, electric guitar amps, and orchestras. We have little
sense of the loudspeakeritself as a musical instnnnent, or of
its essential musical character.However, by physical del'mi-
tion the loudspeakeris in fact a musical instrument,and it has
become, without formal recognition,the predominant musical
instrumentof our time.
In the reproduction of classical music (or any music nor-
reallyperformed acousticallyby tradition and preference),the
generally accepted goal of the sound recording is to simulate
the acoustical performance that was recorded and to evoke its
spirit, ambience and artistic force as compellingly and realisti-
AES 8th INTERNATIONAL CONFERENCE
 THECREATION
tally as possible. In popular recorded music since 1950, no
such goal for the sound recording has existed. In large part,
the music is created for performance by loudspeaker, and any
acoustical performance is normally a simulation. This has sig-
nificant aesthetic implications and represents at least two ma-
jor changes in the way music is created:
- The performance of the music is fixed by the storage
medium and subsequently dispersed (in software form) in
time and space and then generated mechanically by a large
range of different loudspeaker systems;
- The producer or composer exercises a greater control
than ever before over choice of (relative) timbre, tempo, in-
terpretation and mood of the performance and obtains for the
first time ever the ability to create a definitive "best perfor-
mance", while at the same time losing control over the quali-
ty, volume level, playback ambience and gross tonal charac-
ter of the performance,
In the multimack recording, each of an away of tracks usual-
ly carries a signal assumed to be equivalent to a musical pan,
as described above. These parts, in popular music, exist in
families, including the drum kit and electric bass, vocal parts,
other rhythm parts, instrumental solo parts (i.e. "leads"), and
instrumental backgrounds (i.e. "horns .... sweeteners .... string
pads" "keyboards" etc.). Primary musical elements are the
bass (or "kick") dram and electric bass (who together provide
the rhyilunic and harmonic foundation of the music), the lead
vocal, and various instrumental lead solos that occur through-
out the song. The recorded kick dram sound is highly stylized
and dynamically unrelated to the rest of the drum kit. The
recorded electric bass sound is no longer of acoustic origin at
all, but is derived directly from the electrical output of the
bass instrument itself. (Usually, when recording, the bass
player will perform in the control room rather than the studio,
hearing himself via the control room monitors instead of
through a bass guitar amplifier, by preference and established
tradition.) Supporting this basic primary group is a broad ar-
ray of musical elements: the recorded sounds of rhythm gui-
tar(s) and keyboard parts, the rest of the drum kit (especially
the snare drum), wind and string background instruments and
sections, and harmony vocals. These parts are recorded acous-
tically in sections or part by part, and/or sampled or synthe-
sized in whole or in part.
The spatial ambience accompanying such recordings, tech-
nically speaking, consists of discrete delays and reverberance
derived fxom the direct parts, often with delays added to the
onset of reverberance ("predelays"). These delays may be
treated with eq_mliTafion,compression/expansion, and/or am-
plitude-panning in a manner similar to the trealment of the di-
rect voices, as they are passed through similar hardware. The
values pertaining to ambience are established intuitively, and
are an essenfialpart of the sylistic signature of a multimack
producer and/or remix engineer. Separate delays or groups of
delays may be used on individual parts, and separate artificial
reverberances may be similarly allocated, so that a finished
pop/rock mix may include the simulation of multiple rever-
berant environments: one for the lead vocal, another for the
snare drum, a third for the strings, and so on. In the face of
the above treatments and practices, it is clear that the simula-
tion of a distinct, singular, coherent acoustical space and the
AESSthINTERNATIONAL
CONFERENCE
OFMUSICAL
SOUNDS
FORPLAYBACK
THROUGH
LOUDSPEAKERS
placement of direct voices in such a space is irrelevant in pre-
sent-day popular recordings.
Arising from these treatments of direct voices and the lay-
ered ambiences assigned to those voices is an aesthetic of
stereophonic playback that has evolved over the past two
decades with little or no formal notice. This aesthetic (which I
have called the "A+B/A-B aesthetic")8 is de facto and clearly
in response to the desirability of a strong stereophonic illusion
coupled with the need to have that illusion work for the widest
range of possible stereophonic playback environments.
Briefly stated, the A+B/A-B aesthetic is based on the idea
that popular multitrack music consists of two complementary
and interactive realms: the realm of direct musical statement
and exposition, which is represented by the (A+B) compo-
nent of the recording, and the realm of supporting and often
antiphonal rhythmic, harmonic and textural parts, which is
represented by the (A-B) and (A,B) components of the
recording. (The [A+B] component is that group of elements
that is common to both channels A and B, the IA-BI compo-
nent is the group of elements that are common but with dif-
feting signs or polarities, and the [A,B] component is the
group of elements that are present independently in either
channel. For purposes of discussion, IA,BI is usually includ-
ed in the set of [A-B].) The interaction of these realms results
in a clear and musically effective spatial polyphony that is
audible over a broad range of playback systems. This interac-
tion sums effectively to monophony; although the spatial
polyphony is lost, other musical elements remain essentially
intact. The aesthetic arises naturally from the act of mixing
on the median plane with panning-by-amplitude as the pti-
mary determinant of localization.
The aesthetic is comparatively simple in its realization in
pop music: lead parts, kick-drum and eleclric bass are (A+B),
which is to say that they are centered in the monaural phan-
tom image between speakers, and virtually everything else is
either (A-B) or (A,B) in a balanced array, for the purposes of
either "framing" or "answering" the (A+B) part(s). Ambience
is generally and most effectively transmitted in the (A-B)
realm and is used as a textural agent in the recording produc-
don. There is no simulation of the environment of a live per-
formance, no attempt to realistically invoke the illusion of
such an environment. The A+B/A-B aesthetic results in a dis-
tinct musical character, one that is only heard via loudspeak-
ers playing back recordings. It is not heard in acoustic or rein-
forced live performance, nor is there any attempt to simulate it
in those venues. It is, simply, a particular and unique charac-
teristic of music intended for playback over loudspeakers.
A note about spectra of these various components: for rea-
sons originally having to do with limitations of the stylus
movement in the record groove, low frequency (A-B) signals
have been avoided in popular multitrack recording. Further, as
localization of acoustical sources in space is done in signifi-
cant part by simultaneous intra-ear analysis of pinna effects at
the basilar membrane9, the use of high-frequency (A-B) and
(A,B) spectra for spatial localization has proved perfectly ap-
propriate and effective, while at the same time being techni-
cally acceptable for stereophonic record production. The "pre_
ferred" low frequency spatial informationl0,11 that normally
leads to our sense of sonic space (as opposed to source) has
165
MOULTON
been lost and/or not simulated, and sense of spaceis transmit-
ted primarily via reverberant wash. Therefore, a spectral dif-
ferenfiation between (A+B) and the other elements can be
heard.Low frequency content is generallyrestrictedto (A+B),
while (A-B) and (A,B) components are particularly rich in
spectraabove 2500 Hertz.
Incidentally, study of the (A-B) component of a stereo-
phonic recording, (by inversion of the [BI componentprior to
summation to mono) leads to many quite useful insights
about the nature of the recording methods and style of any
given producer. First suggested to me by Bob Ludwig, this
study is highly recommended to any serious student of music
recording, particularly as it applies to popular multitrack
recordings.12
5. THE LISTENING ENVIRONMENTS
It is unreasonable to attempt any assessment of style and
usage of an art and its medium without also a consideration
of the audience(s) for that art. The audiences for recorded
music are extremely diverse. However, it is possible to make
some generalizations about the audience for so-calledpopu-
lar music, and about the conditions under which they listen
to music,
The audience for popular music is comparatively young
(and/or nostalgicfor their youth as a functionof their listening
to music). Their experience with music is predominantly
through recording and broadcast, and involves little, if any,
experiencewith acousticperformances of music. The range of
listening systems/environments which they use is broad and
diverse. The various types of environments may all be regard-
ed as "typical," which leads to the paradoxical and problemat-
ic notion that "ideal" recordings have characteristicsthat are
"ideal" for all of these environments. The general group of lis-
teeingenvironmentsincludesthefollowing:
- The Living Room: typically with two loudspeakersand a
viable median plane, possibly integrated with a monaural
television. The typical residential living room is the basis for
the IEC listening room. The useful range of Sound Pressure
Levels is from ca. 40 to ca. 100 dB SPL, with a "normal"
range probably being 55-90 dB SPL. Extremes of the range
are represented on one hand by the "high-end audiophile"en-
vironmentand the "compact" stereo in a small apartment on
the other,
- The Kitchen/Bedroom: characterized by monaural table
radio/TV.No stereo is availableand no median plane is possi-
ble. Thereis a useful dynamic range from 40 to 75 dB Sound
PressureLevel.
- Headphones:characterized by wide dynamicrange (up to
70 dB in the ear), broad bandwidth and a constant median
plane. Headphones yield comparatively strong stereophonic
illusions and significant audible differentiation between
recordingsof stereophonicand monophonicorigin,
- The Automobile: characterized by the presence of stereo
(often with multiplepairs of speakers, all off-axisto the listen-
ers), no median plane possible, very limited dynamic range
(10-30 decibels maximum) combined with extremely high
backgroundnoise levels (60-100 dB SPL).
-The Boom-box: a portable stereo playback system de-
signed for nearfield individual listening. Usually includes a
166
"stereo-enhance"switchwhich reverses the polarityof one of
the speakers for greater illusion of space in the playback, at
the sacrifice of a stable monaural phantom image. The ambi-
ent noise levels are the same as the range of the levels of the
environmentitself.
The environments mentionedabove are sufficientlydiffer-
ent that they are almost incapable of being reconciled, with
the automobile, the headphoneand the table radio presenting
stereophonic experiencesthat are mutally exclusive.Clearly,
music that is to be successfullyenjoyed in all of these envi-
ronmentsmust:
(a) be comparativelysimple spatially and texturally (which
is not to imply either crudenessor lack of musicalsensibility),
(b) have limited note-to-notedynamicrange,
(c) have a lownoise floor,
(d) haveexaggerated spatialcharacteristics
that donot
come significantly degraded timbrally when summed to
mono,and
(e) not be dependent on the outer octaves (20-80 Hz., and
10-20 ldtz.) of the audible spectrum for musical effect.
Popular music generally fits this template, by economic
necessity. The recording practices currently in use support
such a template, and it is not reasonable to assume any nar-
rower range of end-user environments for the purposes of as-
sessing the desirabilityof any recordingcraft, technique,suit-
ability, or realism, except when specifically considering
.either a "niche" style of recording (such as an audiophile
recording) or a "niche" style of music (such as heavy-metal).
The creators of music for mass distribution can certainly be
expected to continue to create music which remains broadly
suitable for these diverse environments, while perhaps not
ideal for any of them.
6. THE NATUREOF THE MUSICAL VOICE
Music is not a waveform, but a psychological and spiritual
conslruct within the mind. The waveform and its physicaldi-
mensions are simply carders of musical information, while
music itself lies within higher realms of processing in the
mind: the processing of patterns,the mindplay of neural tern-
plates and associations, and the mysteries underlying our
emotional, spiritual and physicalresponses.13This is true for
all music, including that generated by loudpeakers, and it
may be that the best productiongoal for loudspeakermusic is
to create sound artifacts that are as rich in the pattems of mu-
sicalperformance as are musical sounds created in the acous-
ticalrealm.
The basic elements of sound are the pauems of freqencies,
spectra and amplitude that we traditionally associate with
waveform.The higher-levelmusicalelements are the patterns
of change of these basic elements. It is these patterns of
change that our neurological system seems particularlywell-
suited to observe and enjoy and that are central to the idea of
musicality.Another way of puttingit is to say that musicalex-
pressionresides primarilyin the transitionsbetween notes,the
differencesbetween the waveforms, and the dynamic ebb and
flow of intensityand spectrum,ratherthan in the notes, wave-
forms, spectraand intensitiesthemselves.
At both conscious and pre-consciouslevels, our enjoyment
of music is associatedwith our observationof and mentalplay
AES 8th INTERNATIONAL CONFERENCE
 THE CREATION OF MUSICAL SOUNDS FOR PLAYBACK THROUGH LOUDSPEAKERS
with patterns of variation in time, fiequency and localization,
This enjoymentis derived from both the obvious and the inex-
plicable audible realities,and the tension between them. We
enjoy the sound of the musical ensemble, for instance,partly
because we don't hear the many separate voices, but one
greater voice that transcendsthe quality of individual voices,
Misdirection,timbral ambiguity,phantom sources, and meta-
inslmmentsare all normalelements in the craftof Iraditional,
acoustical orchestration. By the same token, we enjoy the
mysteriesof illusionassociated with the (A-B) stereophonic
artifact and the illusions of stereophonic acoustical realities
thatdiffer fromthe physicaland visual ones that actually sur-
round us. Our enjoyment is also based upon the predictable
and unpredictablepatterns of change in micro-, intermediate-
and macro-_s of time: change in shapes of the dynamic
elementsof notes (attacks,releases,etc.), change in shapes of
loudnessand timbre of individual notes, and change in pat-
terns of shapes of loudness and timbre of patterns of
notes.14,15We respondto the implied speechand motor rem-
plates such pattems evoke, and respond to the tensions that
emerge between those templates and the image templates
evoked by lyricsand other associativepropertiesof the musi-
cai setting.
One of the most successful aspects of acoustical-source
sound recordinghas been its ability to accuratelyrecreate the
emotivepatterns generatedby the recorded performersand to
therebytransmitsomethingof the emotionalintensityinherent
in the original performance. A conesponding deficiency of
soundrecordingto date arises from the failureof loudspeakers
to mimicthe way musicalinstrumentsinteract with the perfor-
mance space, thereby reducing the impact on the listener re-
lated to the perceptionof that interactionof source and room.
This deficiency is particularly evident in multitmck recording
of popularmusic,
6A. THE MUSICAL VOICE IN THE ACOUSTICAL
REALM
In the acousticalrealm, the sound of music arrives at our ears
via multiplepaths with a complexity that is,when confronted
with the conventionaltoolsof physicalacousticalmeasure-
ment, both bewildering and baffling. As Arthur Benade has
noted, the musical source is coherent, our perception of that
source is coherent, but the intervening transmission path is
chaotic.16 It is not clear, at this lime, how we reconstruct the
musicalityof the sourceinto such a clear perceptualconstruct
from the chaotic transmissionpath, but it is axiomatic that we
do. Benade furtherobserves,and generalobservationof musi-
dans supports this idea, that the complexity of the Iransmis-
sion path seemsto enhance,not confuse, our perceptionof the
acousticalmusical sound.17 We take for granted the acousti-
cal reality of our musicalperception. Processes leading to the
precedenceeffect integrateour perceptionof room reflections
with our perceptionof the direct sound, so that most of what
we hear appears to be direct sound (even though the intensity
of the reflected sounds is usually equal to or greater than the
intensity of the direct sound), and the early reflections
themselvesare not apparent to us. Reverberance is perceived
as a wash of sound followingthe direct sound,and we tend to
define "good" and "bad" acoustics based upon the perceived
AES 8th INTERNATIONAL CONFERENCE
characterof this wash,rather than in response to the ensemble
of heardbut unnoticedearly reflections.
Listening to music via loudspeakers presents some inter-
esting issues in regard to this process. The high-frequency
emissions from loudspeakerstend to be quite directional,and
our intuitive design tendency has been to suppress reverber-
ant paths in the playback space as well. Further, recording
tradition has emphasized pickup of only direct sound in the
recording studio (the microphone cannot make the use of the
multiple paths that the human hearing mechanism does, and
its summation of the information of those multiple paths
leads to a subsequent degradation of timbre, due to the quite
audible interference effects resulting in comb-filtering). The
result is that musical sound from a loudspeaker usually lacks
the multiple, reflected paths of a typical acoustical musical
sound to the ear and suffers in comparison with such a sound
due to its one-dimensionalsingularity.I speculatethat a large
part of the reason for the commercial success of stereophony
lies in its partial recreation of the multiplicity of paths inher-
ent in the acoustical sound, and consequent enhanced enjoy-
ment of the musical artifact patterns that such multiplicity
seems to provide.
When we artificiallyinvoke the precedence effect with the
use of short delays (less than 50 milliseconds),thereby simu-
lating the presence of multiplepaths, it becomes immediately
apparentthat the presence of the delayed (reflectedpath) acd-
fact has a largeimpact on the subjectivequality of the sound.
Workingwith synthesizersand delay lines quickly leadsto the
insight that synthesizer sounds "come to life" with the addi-
tion of early delays, particularlywhen the delays have differ-
ent localizationcues than the undelayed sound (thus mimick-
ing the directional behavior of direct and reflected sounds in
the acoustical realm), and subsequent removal of such delays
causes the sound to appear fiat and one-dimensional,without
body or depth. (Interestingly, synthesists almost invariably
workwith reverberance added,because, they will tell you,the
dry sounds are simply toodull to put up with for the hours
flat are requiredfor mostsynthesis work.)
6B. THE MUSICAL VOICE INTHE ELECTRICAL
REALM
In contrast to the acoustical source, the electrical audio
waveform is singular, and involves no ensemble of complex,
phase-locked iterations such as we encounter in acoustical
space. The conceptual substitution of this waveform for its
complexand multi-facetedacoustical counterpartis a primary
feature of multilrackrecording. This has lead to the practices
discussed in this paper: the in-series manipulation of the
waveform by equalization and compression/expansionin or-
der to manipulate timbre, and the parallel branches of delay
and reverberation,which are clearly an attempt to reintroduce
the interest caused by the multiplicity of signal paths in the
acousticalrealm. In general,lime effort has been expendedon
the development of a more complex electrical signal model
for recordingproduction, except in the area of binaural sound
and for a varietyof post-productionillusionenhancementpro-
eesses (such as the various quadraphonic matrices and more
recent commercialprocesses).Synthesis developmentcontin-
ues to explore the development of singular algorithms and
167
MOULTON
samples,and the evolution of consoles, signal processorsand
microphonesseems inextricablywedded to the notion of mu-
sic-as-single-waveform,
Much of this has to do with the need to minimize data re-
quirements.Audio is, thanks to the extreme dynamicrange of
our hearing, data-hungry. The great promise of mulfiwack
recording and synthesis (to give the composer/producer
greater control of and facility with music at reduced cost) has
been traditionally limited by data limitations (manifesting
themselves as track number limits and clearly audible band-
width limitations and distortion/noise thresholds). Develop-
meat efforts have been expended primarily upon improving
control and facility, and reducing data needs through time-
sharing, the substitutionof control algorithms and samples for
mw data,and simplified signal processingmodesof operation
(parfio,larly compression, expansion and reverberance)that
provide at least a rudimentary and sometimes a sophisticated
mimicry of the more complex acoustical reality.At the pre-
sent time, the notion of a multiple-iteration version of each
musical voice (thus replicating a basic acoustical reverberant
behavior)is theoretical,
7. CONCLUSIONS
The productionof popular music via multitrack recording
representsby far the great majorityof all music recordingpro-
ductiondone in the world today (perhaps 90% of it).Over the
forty years since the advent of multitrack technology,a style
of multitrack recording has.come into being that is distinct,
particularly in its stereophonic reali?ation, from the styles of
recording employed in the recording of acoustical musical
events. As this paper has described, this style is in large part a
response to comparatively simple models for the musical sig-
hal, the demands of an extremely wide rangeof playback eR-
vironmeats, and a production environment that emphasizes
qualities that are suited to the configuration of theproduction
hardware but not particularlyrelevant to the end-user's play-
back environmeat(s). This has led to the evolved aesthetic
processdescribedemlier.We'are at a point, however,where
both changes in the hardware system and a growing body of
knowledge and historical perspective have led us to begin to
institutechanges in the ways we create this music,
Central to this change is the acknowledgementof the im-
portance of pattem-cbange in musical aesthetics. The devel-
opmeat of more sophisticatedand musical algorithms for se-
queace manipulationand control and for note-to-notetimbral
control in synthesis (to permit the comparativelyeasy integra-
tion of expressive nuance into a group of notes) is beginning
to occur in the realm of music synthesis,and there are a wide
variety of ambience enhancement devices of all types and at
all prices in de facto acknowledgement of the need for sonic
complexityin our musical signal generation.
In the hardware realm, increasing availabilityof memory
for data and and speed for conlrol algorithms arerelaxing the
constraints we have faced more quickly than we can assimi-
late the new capabilities they present. Conceptually,we can
now reconsider some of our technical notions regarding the
nature of music. For instance, the representationof a recorded
or synthesized musical sound as a seven-iteration construct
(each with its own spectrum and angle-of-arrivalspectralsig-
168
nature),to mimic what happens in a conventionalroom, may
prove to be aestheticallyand aurally very satisfyingand it cer.
tainly is a significantly more robust representationof acousti-
carlreality than is our Currentelectronic representation of a
sound. The synthesis of music via the generation and imple-
mentationof templatesof patterns of change relatedto specif-
ic emotional states, gestures and language,overlaid on values
of frequency, amplitude and specman, may prove to be far
more palpable, moving and effective than our present synthe-
sis modalities.
Toward this end, more complex microphone, signal-hah-
dling and signal reproductionsystems might be called for, al-
though I expect two-channel stereo to remain the playback
medium of choice, and (more likely) we will develop elabo-
rate signal-processing algorithms functioning globally in a
console to simulate such systems effectively. Poss_ilities in-
dude the ideas of Middle-Side and Seven-Path Model Syn-
thesis and global reverberators (consoles?)whosearchitecture
permits the simulation of complex acoustical spaceswith the
insertion of multitrackmusical voices at differentiated points
within those virtual spaces. At another level, simulationand
insertion of psychoacousticcues to invoke auditory illusions
(suchaspinnareflectionsimulationto invokeiRma-aural lo-
calization,and non-linearity simulationto invoke pseudo-
loudness)are technologicallyin developmentand limiteduse
now, and almost certainly will be used broadly and freely as
various correlations between psychoacoustic stimuli and re-
sponsesare more fully establishedand theirwindowsof effec-
tiveness defined.
Philosphically,this moves away from the orginal archival
notion of accuracy in sound recording, and toward the notion
of intensityof illusion. Sucha directionis certainly in keeping
with the aesthetic propensitiesof popular music,and it proba-
bly represents a more sensibleoverall goal artisticallyas well:
to produce the most powerful, affectingand compelling musi-
cai art that we can.
ACKNOWLEDGEMENTS
The author would like to thank Dr. Robert Myers,Associate
Dean of Curriculum, and Dr.Thomas Rhea, AssociateProfes-
sor of Music Synthesis, both at Berklee College of Music,
Boston, MA, for their invaluableassistance in the preparation
of this paper.
REFERENCES
1. Davis, Don & Davis. Chips, The LEDE® Concept
' for the Control of Acoustic and Psychoacoustic Parametersin
Recording Control Rooms. Journal of the AudioEngineering
Society28:9, Sept. 1986,pp. 585-595.
2. Muncy,Neil, private conversations,1986-89.
3. Druckman, Jacob, lectures in orchestration (cf.
Berlioz, Stravinsky, Piston), Juilliard School of Music, New
York,NY, 1963-6.
4. Rhea, Tom,private conversation, 1990.
5. Moulton, David, The Perception of Stereophonic
Artifactsin a ReverberantSpace, lecture presentedto the Fre-
donia section of the AES, Fredonia, NY, April 20, 1989.Un-
published.
6. Moulton, Ferralli, Hebrock and Pezzo, Localization
AES8thINTERNATIONAL
CONFERENCE
 THECREATION
of PhantomImagesin an OmnidirectionalStereophonicLoud-
speakerSystem, The, presentedat the Audio EngineeringSo-
ciety81st Convention,November, 1986,Los Angeles,CA.
7. Blauett,Jeus (la'.Allen), SpatialHearing,MIT Press,
Cambridge,MA, 1974,1983,pp. 204-213.
8. Moulton,David, MiscellaneousNotes in Supportof
CriticalListening and Audio Ear Training Presentations, pp.
7-10. NationalPublic Radio Training,Washington,IX21990.
9. Wright, Hebrank & Wilson, Pinna reflections as
cues for localization. Journal of the Acoustical Society of
America56:3, 1974,p. 957.
10. Schroeder,Gottlob & Siebraase, Comparative study
of Europeanconcerthalls: correlationof subjectivepreference
with geometric and acoustic parameters., Journal of the
AcousticalSocietyof America56:4, 1974,p. 1195ff.
AES8111
INTERNATIONAL
CONFERENCE
OFMUSICAL
SOUNDS
FORPLAYBACK
THROUGH
LOUDSPEAKERS
11. Griesinger,David,Spaciousness and Localization in
Listening Rooms and Their Effects on the Recording Tech-
nique, Journal of the Audio Engineering Society 34:4, 1986,
pp. 255-268.
12. Ludwig,Robert, private conversation,1985.
13. Roederer, Juan G., Introduction to the Physics and
Psychophysics of Music, 2nd Ed., Springer Verlag, NYC,
1973, 1979,pp 11-12,161-170.
14. Meyer, Leonard B. Emotion & Meaning in Music,
The Universityof ChicagoPress, Chicago, IL. 1956.P.22-43.
15. Roederer,Juan G., pp. 3-5.
16. Benade, A.H., From Instrument to Ear in a Room:
Direct or viaRecording,Journal of the AudioEngineeringSo-
ciety 33:4, April, 1985,pp. 218-9.
17. Benade,A.H., pp. 226.
169