Hearing a mistuned harmonic in an otherwise periodic complex

tone a)
William Morris Hartmann
Physics
Department,MichiganStateUniversity,
E. Lansing,Michigan48824andInstitutdeRecherche
et
CoordinationAcoustique/Musique,
31, rueSaint-Merri,F-75004,Paris,France

StephenMeAdams
") andBennettK. Smith
InstitutdeRechercheet CoordinationAcoustique/Musique,
31, rueSaint-Merri,F- 75004,Paris,France

( Received12March 1990;acceptedfor publication19June1990)

The ability of a listenerto detecta mistunedharmonicin an otherwiseperiodictone is
representative of the capacityto segregateauditoryentitieson the basisof steady-statesignal
cues.By useof a taskin whichlistenersmatchedthe pitchof a mistunedharmonic,thisability
hasbeenstudied,in order to find dependences on mistunedharmonicnumber,fundamental
frequency,signallevel,and signalduration.The resultsconsiderably augmentthe data
previouslyobtainedfrom discriminationexperiments and from experiments in whichlisteners
countedapparentsources. Althoughpreviouswork hasemphasized the roleof spectral
resolutionin the segregation
process,the presentwork suggests that neuralsynchronyis an
importantconsideration;our data showthat listenerslosethe ability to segregatemistuned
harmonicsat high frequencieswheresynchronous neuralfiring vanishes.The functionalform
of this lossis insensitiveto the spacingof the harmonics.The matchingexperimentalso
permitsthe measurementof the pitchesof mistunedharmonics.The data exhibitshiftsof a
form that arguesagainstmodelsof pitch shiftsthat are basedentirely upon partial masking.
PACS numbers:43.66.Fe,43.66.Hg,43.66.Jh,43.66.Ba [WAY]

INTRODUCTION gatorsfor tones(McAdams1984a,b, 1989), butperhapsnot

In everyday circumstanceslisteners are exposedto for speech(Gardner and Darwin, 1986).
sound fields that have been created by several different There are, in addition,cuesto segregation presentin
acousticalsourcesacting simultaneously.Therefore, a nec- steady-statesignals.Importantamongtheseis the harmoni-
essaryaspectof humanhearingis the ability to separatea city or inharmonicityof the spectralcomponents. When
compositesoundfield into distinctauditoryimagesor per- spectralcomponents havefrequencies that are in a perfect
ceptualentities,normally correspondingto the severaldif- harmonicrelationship,thosecomponents tend to be inte-
ferent sources(Moore, 1982, Chap. 6; McAdams, 1984a, gratedbytheauditorysystem intoa singleperceived entity.
1989). Becausethe channelsof the auditory systemare The presence of oneor moreinharmoniccomponents in a
tuned,it wouldbenaturalfor thesystemto segregate entities spectrumpromotesthe segregation of two or moreentities.
that are entirelydistinctin their frequencyranges.But as A simplepsychoacoustical approachto thestudyof seg-
Plomp (1976, p. 146) has noted,the systemis frequently regationbaseduponinharmonicityis the mistunedharmon-
required to segregatesourceswith spectrathat are inter- ic experiment.Here, the listeneris presented with a complex
leaved.It is evidentthat auditory processesmore intricate tonecomposedof many harmonics,but oneharmonicis mis-
than simplefilteringare involved.A reviewof someof these tunedfrom its correctharmonicfrequency.If the amountof
processes andthe corresponding acoustical
cueswasgiven the mistuningis largeenough,the listenerhearsa sinetone
by Hartmann (1988). asa distinctentitysegregatedfrom the lower-pitched com-
In segregating thephonemes of speech(part of thecock- plex tone.The pitch of the segregated entity corresponds
tail party effect) or the tonesof individual instrumentsin approximatelyto the frequencyof the mistunedharmonic.
polyphonicmusic,asynchronous onsetsprovidea majorcue Early mistunedharmonic experiments(Moore et al.,
to the auditorysystem(Bregmanand Pinker, 1978;Rasch, 1985;Hartmann, 1985) were discriminationexperiments.
1978). Other temporaldifferences amongsources, e.g.,slow The listenerheardtwo tones,one of them with perfectly
amplitudeor frequencymodulation,are alsoeffectivesegre- harmoniccomponents,the other with a mistunedharmonic.
The listenerwas requiredto say which of the toneswas in-
harmonic.
Some
ofthecontents
ofthisarticlewerepresented
atthe112thMeeting
of The discrimination experimentis straightforward
and
theAcoustical
Societyof Americain Anaheim,CA [J. Acoust.Soc.Am.
Suppl. 1 80, S93 (1986)].
efficient.Unfortunately,the resultsof thisexperiment
are
Present
address:
Laboratoire
de Psychologie
Exp6rimentale
(CNRS difficultto interpretbecausethelistenercanactuallymake
URA 316), Universit6Ren6Descartes
(EPHE), 28 rue Serpente,
F- judgmentson the basisof a numberof differentcues.Hart-
75006 Paris, France. mann (1988) identifiedfive suchcuesto inharmonicity;

1712 J.Acoust.
Soc.Am.88(4),October
1990 0001-4966/90/101712-13500.80© 1990Acoustical
Society
ofAmerica 1712
their relativesaliencedependsuponthe amountof mistun-
inganduponthestimulus duration. Someofthesecues,such
as a shift of the low pitch or roughness, do not seemto be
relatedto the segregation of a mistunedharmonicasa dis-
tinct entity.Therefore,whilethedetectionofinharmonicity,
as revealedin discriminationexperimentsmay be of some
interestin itself,the datafrom theseexperiments cannotbe
applieduncriticallyto thequestionof segregation of entities.
This conclusionwas also reached by Moore et al.
(1986) whentheyrepeatedtheir 1985experiments, thistime
askinglisteners to ignoreall othercuesandto respondposi-
tivelyonlywhentheyweresurethat theyheardout the mis-
tunedharmonic.Not surprisingly, the resultsof thissecond
seriesof experiments differedfrom thefirst.Thresholdswere
higherand the dependence upon mistunedharmonicnum-
ber wasmuch lessapparent.
The goal of the experimentsdescribedin the present
article was the same as in the discriminationexperiments
describedabove,namely,to probethe limits of a listener's
ability to hear out a mistunedharmonic.The method,how-
ever,was different.The presentexperimentsattemptedto
ensurethat a positiveresponse occurredonly whenthe lis-
tenerindeedheardout themistunedharmonicby askingthe
listenerto matchthe pitchof the segregated
component.If
thelistenercorrectlymatchedthefrequencyof themistuned
harmonic, then that was taken asevidencethat the mistuned
harmonicwassegregated becausethe probabilityof making
a correctmatchby chancewassmall.
It was expectedthat this procedurewould give results
that are more reliable than either of the experimentsby
Moore et al. This alsomade it possibleto extendthe mea-
surementsto higher harmonic numbers(higher than the
6th), where interestingeffectswere expected.Further, we
hopedto eliminatethe considerableintersubjectvariability
seenby Moore and his colleagues.
I. METHODS
A. Procedure
The listener was seated in a double-walled sound-treat-
edroom(SolunaSN 1). Beforehim wasa response boxwith
threepushbuttons,threelamps,andthreepotentiometers.
When the listenerpushedthe greenbutton,the computer
choserandomlya particularfrequencyfor the fundamental
of thecomplextone(randomizedaccordingto a rectangular
distribution with a width of ñ 10% about the nominal fun-
damental)anda particularharmonicof the complextoneto
be mistuned.When the listenerthenpressedthe yellowbut-
ton, he heardthe complextone;whenhe pressedthe orange
button, he heard a sinetone. Each tone was precededby a
delay.The frequencyof thesinetonecouldbecontrolledby
two ten-turnpotentiometers,onea coarsecontrolcovering
therangefrom0-16 kHz, theothera finecontrol,whichwas
ten times less sensitive.
The listener's task was to adjust the potentiometers so
that thepitchof thesinetonematchedthepitchof the har-
monicheardout of the complextone.Therewasno limit to
the number of times that the listener could hear either tone.
With thethird potentiometer,
a fader,thelistenercouldad-
1713 J. Acoust.Soc.Am.,Vol.88, No.4, October1990
justthelevelof thesinetoneforhisownconvenience.
When
thelistenerwassatisfied with hismatch,heagainpressed the
greenbutton,and the trial with a particularfundamental
frequencyandmistunedharmonicwasfinished.At theend
of a trial, the listenerreceivedfeedback,a whitelamp if his
match was within ñ 0.8 harmonic number of the correct
value,a redlampif hismatchwaslowerthanthisrangeanda
blue lamp if higher.
Each experimentalrun consistedof 12 or 16 trials
(matches)and on eachtrial a differentharmonicof the com-
plextonewasmistuned.The mistuning,measured asa per-
centage of theharmoniccomponent frequency, wasthesame
for eachmistunedharmonicthroughoutthecourseof a run.
Runs typicallylasted10 min after which the listenercould
come out of the sound-treated room and rest.
B. Listeners
The three authors B, S, and K servedas listeners.They
rangedin agefrom 33-47 years;all weremaleswith normal
hearingaccordingto their own reports,and all wereexperi-
encedin psychoacoustical
tasks.They had musicalexperi-
enceandcouldsingtheharmonicseriesup to the tenthhar-
monic (jumping octavesof course) or, in one case,the
eighth.
C. Stimuli
Mainly, thestimulifor thisexperimentwerea subsetof
thoseusedby Mooreet al. (1986). The nominalfundamen-
tal frequency was200or 400or 800Hz, andthedurationof
thecomplextoneswas410or 50 ms.Therewere16harmon-
icsof equalamplitudein thecomplextonespectrum, except
thattherewereonly 12whenthefundamentalfrequencywas
800 Hz. The low-frequency harmonicswerecandidates for
mistuning:thefirst 16for a fundamentalof 200 Hz, thefirst
12 for fundamentalsof 400 and 800 Hz. The proportional
frequencydeviationfor mistunedharmonicswasconstant
fora givenrunandvariedparametricallybetween0.5% and
8%. All the harmonics,includingthe mistunedharmonic,
startedat a positive-going
zerocrossing(sinephase).The
complextonehada standard levelof 60dB SPLpercompo-
nent;for low-levelrunsthelevelwasreducedto 30 dB SPL
per component. Both equalizedand nonequalized tones
wereused(seetheAppendix).Toneswereturnedonandoff
with a raised-cosine envelopeof 10-msduration,and the
delaypreceding eachtonewas300msfor 410-mstonesand
200 msfor 50-mstones.The matchingsinetonehad a dura-
tion of 300 ms, and it wasdelayedsimilarlyto the complex
tones.
Thestimuliweregenerated
bythe4C digitalsynthesizer
at the Institut de Rechercheet Coordination Acoustique/
Musique(Moorer et al., 1979). The synthesizer wascon-
trolledbya PDP 11/34computer, whichalsorantheexperi-
ment. Stimuli were convertedto audio by Tim Orr 16-bit
DACs at a samplerate of 32 kHz, andwerelow-passfiltered
from 12.8kHz ( -- 96-dB/octslope).The listenerheardthe
sounddioticallythroughBeyerDT-48 headphones with fiat
foam cushions.Signallevelswere measured 'at the head-
phoneswith a BruelandKjaer 2209soundlevelmeter(A-
Hartmann
et a/: Mistuned
harmonic 1713
weighted)witha fiatplatecoupler.Thecomputer collected rameters,
the responses anddid somepreliminarydatareductionso mistuned
that the listener could seehis resultsafter the end of the run.
It storedthe datafor subsequent analysis.
D. Analysis
Therawdataforasetofruns canbeseen ona"matching
plot," as shownin Fig. 1. For a givenmistunedharmonic Second,
(horizontalaxis), theplotshowseachmatch,expressed ona One needsa criterionfor declaringa matchto be a hit or a
continuous scaleof harmonicnumber,obtainedby dividing miss.After someexperimenting,
the listener'smatchingfrequencyby the fundamentalfre-
quencyfor the trial. Figure 1 showsthe data for listenerB
whenthe mistuningwas4%. Althoughit mayappearthat
only a few points are plotted for low harmonic numbers
(e.g.,number1), thereareactuallytenpoints,onefromeach
runof theexperiment. Thestraightlineshowsidealizedper-
formance,matchingfrequencies identicalto mistunedhar-
monicfrequencies. Therefore,thislinehasa slopeof 1.04.
In principle,furtheranalysis
of thedataisa simplemat-
ter.If thematchingfrequency iscloseto thefrequencyof the
mistuned harmonic, then the match is called a "hit." If the
matchingfrequencyis far from the frequencyof the mis-
tuned harmonic, then the match is called a "miss." From the
patternof hitsandmisses
for the variousexperimental
pa-
onecandeducethelistener's abilityto segregate
harmonics froma complextone.
In practice,theanalysis isnotsosimple.Thereisfirsta
problemin identifying a hit, becausepitchshifteffectsare
expected for the harmonics of a complextone (Terhardt,
1971). Thesystematic departure ofthedatapointsfromthe
straightlinein Fig. 1showsthatsuchshiftsindeedoccurred.
thedatashowa continuum ofmatching frequencies.
we establisheda twofold
criterion. An absolute criterionrequiredthat a matchbe
within one harmonic number of the mistuned harmonic
( 1/2 harmonicnumberfor mistuned1and2) in orderto bea
possible
hit. Thenextrequirement
wasbaseduponcluster-
ingof data,i.e., uponself-consistency.
For thematchesthat
survivedtheabsolute criterion,a meanandunbiased(N -- 1
weight) standarddeviationwerecalculated.If the standard
deviationwasgreaterthan 2.5% of the mistunedharmonic
number,then the data point that differedmost from the
mean was discarded and a new mean and standard deviation
werecomputed.The procedurewasrepeateduntil the2.5%
clusteringcriterionwas met or the data setwas exhausted.
Figure2 shows theresultsofapplying
thetwo-stagecriterion
to thedataof Fig. 1.Hitsareshownby -4-symbols; misses
areshownby circles.Clearly,theperformanceof listenerB
for 4% mistuningwas high, at leastfor low harmonic
numbers.

20 Figure3 showsa matchingplotfor listenerB whenthe

mistuningwas only 0.5%. The number of hits was much
18

16
20 ß

18
z
o
:• 12 16

10
14
z
z
3:8 o
12

6 T
10

2
6

0o 2 4. 6 8 10 12 14 16 4

MISTUNED HARMONIC
2

FIG. 1. Plot of the matchingharmonicnumberas a functionof the mis-

tunedharmonic
numberforlistener
B.Thecomplex
tonehad200-Hznomi- 0o 2 4 6 8 10 12 14. 16
nalfundamental
frequency
and16harmonics,eachata levelof60dBSPL.
Thetonedurationwas410ms.Themistuned
harmonicwasmistuned
by MISTUNED HARMONIC
4%. For eachmistuned
harmonic
thereweretenmatches
(symbolq- )
because
thereweretenruns.Thesolidlineshowsidealperformance
where
thematchingfrequency
isidentical
to thefrequency
of themistuned
har- FIG. 2.ThedataofFig.I (4% mistuning) arereplotted
withhits( q- ) and
monic.The slopeof that line is, therefore,1.04. misses(¸) identified
according
to thecriteriadescribed
in thetext.

1714 d. Acoust.Soc. Am.,Vol. 88, No. 4, October1990 Hartmannoral.: Mistunodharmonic 1714

 2O 100 ,
90
f8
80
16
70
oo oO õ
14 60

0 50
'• 12
+•+o
4O
10
3O
Z

-r
(_•
8 20

10
'• 6
o ø
o
00 • 4• ' 6• , 8• ß10• ß12I , 14I , 16r
4 o/0 oo o Mistuned Hormonic Number

•2 o/ooo
/., .o., o,.o.... , .... F[G. 4. Hit ratesaveragedoverthe threelistenersfor a nominalfundamen-
00 2 4 6 8 10 12 14 16 tal frequencyof 200 Hz, a levelof 60 dB percomponentanda toneduration
MISTUNED HARMONIC of 410 ms. Equalizedand unequalizedstimulihavebeenaveragedbecause
no importantdifferencebetweentheseconditionscouldbe found.The pa-
rameteristhepercentage of mistuning,4%, 2%, or 1%, aslabeled,andthe
symbol0 indicates0.5% mistuning.

FIG. 3. SameasFig. 2, exceptthat themistuningis0.5%.

a hit rate of 60% as the threshold.Normally, it wasneces-

reduced.Thereare interestingclusterings
of misses:The lis-
tener often matchedthe 2nd harmonic when it was actually
the 1stor the3rdor the4th thatwasmistuned.The matching
plotshowsthatmistuningthe8thor the 11th harmonicpro-
duceda denseduster of missesin the vicinity of harmonic
9.5. There is anotherclusterof missesnear the top of the
spectrumbetweenharmonics15and 16. It isclearthat there
areeffectsin thedatathat arenot preservedby simplycount-
ing the numberof hitsandmissesfor eachmistunedharmon-
ic. However, this simpleenumerationdoesserveto make
further quantitativeanalysisstraightforward.
II. HIT RATES AND THRESHOLDS
From plotsof hits and misses,as shownabove,it was
easyto computethe percentage of hitsto makeplotscalled
"hit-rate"plots.Theseshowthepercentage of hitsasa func-
tion of the harmonic number of the mistuned harmonic. A
differenthit-rateplot is requiredfor eachdifferentpercen- genuineidiosyncraticperceptualeffects.ListenerB, for in-
tageof mistuning.Figure4 showsa hit-rate plot with differ- stance,had difficulty detectinga mistuned4th harmonic
ent mistunings(4%, 2%, 1%, and 0.5%) superimposed. whateverthe fundamentalfrequencyor signallevel.That
The hit ratesshownthere havebeenaveragedover the three
listeners.
A "threshold"plot ismadefromthe setof hit-rateplots.
It showsthe percentage mistuningthat is requiredto reach
somearbitrarily definedhit rate. It too is plotted against
mistuned harmonic number. We found it convenient to take
saryto interpolatebetweenpercentages of mistuningfor the
60% hit-rate point to determinea thresholdmistuning.
A. Effect of m,stuned harmonic number
Thresholdplots for individuallisteners,B, K, and S,
were obtainedfor a 200-Hz fundamentalfrequencyand a
levelof 60 dB per component.This wasdoneby interpolat-
ingamonghit-rateplotsfor 4%, 2%, 1%, and 1/2% mistun-
ing.The resultingthresholdplotsshowedconsiderable indi-
vidualdifferences.ListenersB andK had higherthresholds
for harmonics 4 and 8 and lower thresholds for odd num-
beredharmonies.This behaviormight indicatesomespecial
perceptualcharacterfor octaverelationships.
However,lis-
tener S had low thresholds for mistuned harmonics 4 and 8
andhigherthresholds
for someoddharmonies
suchas5 and
9.
We believe that the individual differences observed in
our experimentsare not due to randomerror but represent
there are individual differences of this kind is consistent with
the results of Moore et aL ( 1985, 1986). However, one can-
not discussthem in termsof a generalmodel.Therefore,to
make Fig. 5, we averagedthe three individual threshold
graphs, realizing that some potentially interestingeffects
would be averagedaway.

1715 J. Acoust.Soc.Am.,Vot.88, No. 4, October1990 Hartmannota/.: Mistunedharmonic 1715


4 threshold, but, for the mistuned 7th, there were two listeners
z 3
4. 6 •' 8 10 12
MISTUNED HARMONIC
FIG. 5. The closedcirclesshowthethresholdfor detectinga mistunedhar-
monic,averagedoverthreelistenersfor 200-Hz fundamental,a levelof 60
dB percomponent anda tonedurationof410ms.Theerrorbarsare2 s.d.in
overalllength.Theopensymbols showdatafromthestudies ofMooreatal.,
squares
for 1985andtriangles
for 1986.
The closedcirclesin Fig. 5 give the averagethreshold;
the error barshavea total lengthequalto twicethe unbiased
standarddeviation(N'-- I = 2 weight).Error barsare large
becauseof the individual differences.•
Also plottedin Fig. 5 are the data from the studiesby
Moore etal., 1985 (squares)and 1986 (triangles). Because
our criterion for threshold (60% hit rate) was rather arbi-
trary, there is little point in comparingthe absolutevalues
amongdifferentstudies,exceptto notethat theyareapproxi-
mately the same.One can, however,comparetrends,and
thisisdonebelow,in orderof increasing mistunedharmonic
number.
There is generalagreementthat thresholdsdecreaseas
the mistunedharmonic takes on valuesn = 1,2, and 3, with
the thresholdfor the mistunedfundamental(n = 1) par-
ticularlyhigh.Subjectively,the caseof the mistunedfunda-
mentalis unique.It is oftendetectednot somuchbecauseit
standsout as a separateentity but becauseit soundsout of
tune. We are inclinedto regardthe higher thresholdseenfor
the mistuned fundamental as reflectinga qualitative differ-
ence,not merelya quantitativedifference.
As the mistuned harmonic number increases from 3 to
highernumbers,thethresholdisapproximately constantun-
til the dip at the mistuned7th harmonic.In part, this dip
representsa flaw in our experimentaltechnique:Sometimes
a listener'shit ratewasappreciablygreaterthan the thresh-
old valueof 60% evenwhenthesmallestmistuning(0.5%)
was used. In such cases (there were 6 of them out of 48
threshold values), the threshold for that listener was taken
1716 J. Acoust. Soc. Am., VoL 88, No. 4, October 1990
to be zero. Usually, this had little effecton the average
for whom this occurred, and the threshold for the third lis-
tener was also rather low. Therefore, the threshold for the
mistuned7th is not correctlyevaluatedby our procedure.
We can only saythat the thresholdis unusuallylow.
Exceptfor the dip at 7, the thresholdappearsto be es-
sentially constant as the mistuned harmonic number in-
creasesfrom 3 to 11. This behaviordisagreeswith the de-
creasingtrendseenin the 1985dataof Moore etal. It is hard
to makethe comparisonwith their 1986databecausethese
extendonly to the sixthharmonic,but in any case,the 1986
data do not showa decreasingtrend. Together,the results
areconsistentwith theideathat the 1986studyandthepres-
ent studycorrectlyrepresentthe thresholdfor segregating a
mistuned harmonic, and that the 1985 data show thresholds
for detectionof inharmonicitybaseduponbeatsand rough-
ness.
For mistuned harmonic numbers 12, 13, and 14, the
thresholdbecomesquite high. The thresholdvaluefor the
mistuned13th is only approximatelycorrectbecausehit
rates for two of the three listeners were somewhat less than
60% for the largestmistuningused(4%). The threshold
estimateof 4% givenin the figureis, therefore,somewhat
lower than the true value.
For mistuned harmonics 15 and 16 the threshold is
againsmall. One naturally wondersaboutthe origin of this
nonmonotonic behavior. Because harmonics 15 and 16 were
at the top of the spectrum,it seemedpossiblethat the low
thresholdwasan artifactof our experimentalstimuli.Con-
ceivably,it wasan edgepitch (Klein and Hartmann, 1981;
Moore and Glasberg, 1989) As a test of this idea, one of us
(listenerB) did an experimentwherethe complextonehad
24 harmonicsof which the lowest20 couldbe mistunedby
2%. The data againshowedgoodperformancefor mistuned
low harmonics,poor performancefor the mistuned 13th
harmonic,followedby goodperformancefor the mistuned
15th. Performance for 14 and 16 was intermediate. This re-
sult suggeststhat we should accept the low threshold for
harmonics15 and 16 as genuine,and not an artifact.
These peculiar nonmonotonicdata suggestthat two
modesof perceptionmay beinvolvedin the task,their rela-
tiveimportancechangingastheharmonicnumberchanges.
There are two otherpiecesof evidencethat tendto support
this idea. First, introspectivelistening (with the listener
writing notes while unaware of which harmonic is mis-
tuned) agrees:for mistunedharmonicnumberslessthan 12
the mistunedharmonicis segregated asa separatetone.For
mistunedharmonicnumbersgreaterthan 13 there is a beat-
ing sensation, but the beatingtakesplacein a regionof tone
height that is narrow enoughthat one can often make an
acceptable pitchmatch.2
Theotherpieceof evidence fora second modeof percep-
tion is the fact that, for harmonics15 and 16, the expected
dependenceon percentageof mistuningbreaksdown. It
seems thatit isnoeasierto detecta mistuningof4% thanit is
to detecta mistuningof 1% or 0.5%; in fact, Fig. 4 suggests
thatthereverseistrue.Thisunusualdependence arguesfora
differentmodeof detectionfor the higherharmonics.It is
Hartmann et al.: Mistuned harmonic 1716
easyto applythisideato the 16thharmonicbecause itsonly
neighboris the 15th.At thesefrequenciesthecriticalband-
widthis about500Hz. Usingthe rule (PlompandSteene-
ken, 1968) that roughness for two sinetonesdecreases as
theirspacing increases
beyond1/4 criticalbandwidth( 125
Hz), oneexpectsthat 0.5% mistuning(3000 and 3216Hz)
might soundsomewhatrougherthan 4% mistuning(3000
and 3328 Hz).
Our conclusionis that our experimentcan successfully
measurethe thresholdmistuningrequiredfor a listenerto
hearout a mistunedharmonicin a 200-Hz complextoneand
that this thresholdis roughlya constantpercentage of the
mistunedharmonicfrequencyas the mistunedharmonic
numberincreasesfrom 2-11. Beyondmistunedharmonic
number11, the thresholdfor segregation increases
rapidly.
We interpretthe low thresholdsactuallyobservedfor har-
monicnumbers15 and 16 asresultingfrom localroughness
andnot indicativeof segregationof a singlecomponentonly.
B. Effect of fundamental frequency
The effectof thefundamentalfrequencyon theabilityof
listenersto hear out a mistunedharmonicwas studiedby
comparinghit ratesfor givenfractionalmistunings,without
reducingthe resultsto thresholdplots.The threelisteners
eachdid tenrunswith4% mistuningfor eachof threefunda-
mentalfrequencies, 200, 400, and 800 Hz. The averageover
listenersis shownin Fig. 6, with fundamentalfrequencyasa
100
80
7O
6O
,5O
4O
3O
20
10
%,
J 0.8 •.6 2.4 3.2 4.0
MistunedHarmonicFrequency(kHz)
FIG. 6. Hit ratefordetecting
a mislunedharmonic,mistuned
by4%, aver-
agedoverthreelisteners.Thelevelwas60 dB percomponent andthetone
duration was 410 ms. The fundamental frequency is the parameter: circles
for 200 Hz, trianglesfor 400 Hz, and diamondsfor 800 Hz. We suggestthat
the circlesat 2.8, 3.0, and 3.2 kHz shouldbe discountedin comparingthe
plots,because matchingto theseharmonicsof 200 Hz is presumablymedi-
atedby a process that is differentfrom the process
involvedfor lowerhar-
monics of 200 Hz and for all harmonics of 400 and 800 Hz.
1717 J. Acoust. Soc. Am., VoL 88, No. 4, October 1990
parameter.The horizontalaxis givesthe frequencyof the
mistuned harmonic, not the harmonic number. Therefore,
data pointsfor a fundamentalof 400 Hz (triangles3are
spacedtwiceaswidelyaspointsfor 200 Hz (circles).
The closedcircles,indicating200Hz, showthepeculiar
nonmonotonic behaviornearthe 15thharmonicasexpected.
Hit ratesfor the other fundamentalfrequencies
seemto de-
creasemore regularly asthe mistunedharmonicnumberin-
creases. The closed diamonds for 800 Hz show the consider-
able difficulty of this condition;only mistunedsecondand
third harmonicscouldbe reliably detected,evenat 4% mis-
tuning.Figure7 showssimilarresultsfor 200- and 400-Hz
fundamentalsand2% mistuning.The plotsin Fig. 8 for 1%
mistuning
havethesametrendasthosein Fig.7, butall hit
rates are reduced.
Figures6-8 invitecomparison withthreshold graphsby
Mooreetal. (1986) for fundamentalfrequencies of 100,200,
and 400 Hz, where the horizontal axis was chosento repre-
sent mistunedharmonicnumber. If one imaginesthat the
segregation of mistunedharmonicsdependsmainly upon
auditoryfilteringwitha bankofconstant-Qfilters,thenthat
way of plottingthe data makesgoodsense.Sucha plot
shouldproducea universalcurvein that the threshold(or
hit rate) shouldbe independent of fundamental frequency,
i.e.,independent of thefrequencyspacing of theharmonics.
On the other hand, if oneimaginesthat the segregation of
mistunedharmonics dependsmainlyuponneuralsynchrony
andis limitedasneuralsynchronydisappears for increasing
frequency,thenthe kind of plot that we havechosen,with
frequency ashorizontalaxis,is morelikelyto leadto a uni-
versalcurve.The datain Figs.6-8 supportthe latter inter-
pretation.Theysuggest thatsynchrony vanishes ratherab-
ruptly as frequency
increases
from 2 to 3.5 kHz. This
loo
90
8o
70
60
50
40
30
4.8 20
10
0.8 1.6 2.4 3.2 4.0 4.8
Mistuned HormonicFrequency(kHz)
FIG. 7. SameasFig. 6 exceptthat the mistuningwas2% andthereare only
two fundamentalfrequencies,200 and 400 Hz. Circles at 3.0 and 3.2 kHz
shouldpresumablybe ignoredin the comparisonwith the 400-Hz data.
Hartmann ot a/.: Mistuned harmonic 1717
frequencylimit for usefulsynchronyis lower than the upper
limit of 5 kHz basedupon physiologicaldata in cat (John-
son, 1974). Our limiting value does seem consistentwith
Plomp's(1967) psychephysicallimit baseduponthe beat-
ing of mistunedconsonances.
For instance,one doesnot
hear beats between sine tones at 2000 and 3000 Hz, even
though beatsat a 2 to 3 ratio are easilydetectedin lower
frequencyranges.Unfortunately,the data of Moore et al.
(1986) do not contributeon this point becausethey go no
higherthan 2.4 kHz.
C. Effect of signal level
The overalllevelof the signalcanbe expectedto affect
the spreadof the neuralexcitationpatternalongtenetopic
coordinates. If the ability to segregate
a mistunedharmonic
dependscriticallyuponthe spatialdistributionof excitation
for the mistunedharmonicrelativeto neighboringharmon-
ics,onemightseea leveldependence in thehit rate.To test
thisidea,thelistenersdid runsat a levelof 30 dB percompo-
nent to be comparedwith the standardrunsat a levelof 60
dB per component.Comparisonswere made for a funda-
mentalfrequencyof 200 Hz andsignaldurationsof 410 and
50 ms, and for a fundamental of 400 Hz at a duration of 410
ms.
The resultsof theexperimentfor 200 Hz and410 msare
shownin Fig. 9. This figureincludesall the availabledata,
from blocksof runs with 1% and 2% mistuningand for
equalizedand unequalizedspectra(See the Appendix), a
total of eightblocksin all. The scatterplot in Fig. 9 showsa
differenceof hit rates,specificallythe hit rate at the 60-dB
level minus the hit rate at the 30-dB level. In those cases
where hit rates for both levels were 100% or 0%, no data
point wasplotted.
The figure showsthat it was more difficult to hear out
the mistuned fundamental at the low level, a fact that finds a
trivial explanationin termsof equal-loudness contours.At
30 dB a 200-Hz tonecan barelybe heard.
The figure also showsthat there were a few mistuned
harmonicsthatcouldbeheardoutbetterat thelowersignal
levelthan at the highersignallevel.Thesewerethe 4th and
5th. For example,hit ratesfor 1% mistuningof the4th har-
monicaveraged 60% at 60 dB percomponent but 86% at 30
dB percomponent
(410-mstone),a difference
of -- 26%.3
This peculiarleveleffectfor the mistuned4th harmonicwas
magnifiedfor 50-msduration (no figureshown), where the
differenceaveraged-- 70%. A similarlevel effectfor mis-
tuned4th harmonicsin brief toneshasalsoappearedwith
great strengthin discriminationexperiments(Hartmann,
1986).
By contrast,the detectionof mistunedharmonicsin the
vicinityof the 7th harmonicwaseasierfor the higherlevel
tone.For 410-mstonesand 1% mistuningthe differencein
averagehit ratesfor a mistuned7th wasabout20% favoring
the 60-dB levelover 30 dB. A similarplot madefor a funda-
mental frequencyof 400 Hz and duration of 410 ms showed
againthat hit ratesfor the 5th harmonicwerehigherat 30 dB
thanat 60 dB, but that hit ratesfor highermistunedharmon-
ic numberwerehigherat 60 dB.
Theseresultscan be comparedwith the predictionsof
anymodelin whichdetectionisbaseduponoverlapof exci-
tationpatternscreatedby neighboring harmonies.If excita-
tion patternsexpand with increasinglevel (Whitfield,
1967), thenthe comparison leadsto a paradoxicalresult.
Considerthemistuned7th harmonic:We findexperimental-
ly that it is more easily detectedat 60 dB than at 30 dB.

100

100
80 o
oo o
60
o o o o
40 o o oO O
o ooo o o
2O
0 OO O•OOOOO
60: 0
o8o • oo
o--o•-o-o--o o o
5O OOOo• o o o
-20

40
o oo o oooõ
o o oo
-40 o õo o o
o
30
-60
20
-80

lO
-1000 ß • ß 4I ß • ß 8I I 10I " 12I • 14I I 16I
•'.o..... 0.8 Mistuned Harmonic Number
Mistuned Hormonic Frequency(KHz)
FIG. 9. Each circlerepresents
the differencebetweenthe hit rate in runs
wheretheIcyelwas60dB/component
andcorrespondingrunswherethe
FIG. 8. SameasFig.7 exceptthatthemistuning
was1%.Thefigureshows levelwas30dB/component.
Whenthedifference
ispositive,
performance
that hit rates are similar for 200 and 400 Hz. Circles at 3.0 and 3.2 kHz wasbetterat the60-dBlevel.Somecircleshavebeenshiftedlaterallysothat
shouldpresumably
beignoredin thecomparison
with the400-Hz data. no circlesoverlapentirely.

1718 J. Acoust. Sec. Am., Vol. 88, No. 4, October 1990 Hartmann eta/.: Mistuned harmonic 1718
Presumably,thismeansthat detectionof the mistuned7th is Our data showed that the hit rates for 50-ms tones were
aidedbyincreasingthewidthof theexcitationpatterns;
at 30 lessthan those for 410-ms tones. This tended to be true for all
dB, theamountof overlapissuboptimal.If thisargumentis mistunedharmonicnumbers;it wasespeciall:/true for the
accepted,thenthe predictionfor the mistuned4th harmonic low-frequencyharmonics1 and 2. Such a reductionis of
is clear. Near the 4th harmonicthe excitationpatternsareparticularinterestif one supposes that the detectionof a
evenmorewidelyseparated(on a physiologicalscale) than mistunedharmonicdependsupon time-dependentfeatures
in the vicinityof the 7th. Therefore,oneexpectsthat again
of the waveform.The elemental temporal feature in the
detectionperformancewill be betterfor increasedlevel.Ex-physicaldescriptionof the stimulusis the dephasingof a
perimentally,however,thereversebehaviorisobserved. Ac-mistunedharmonicwith respectto otherharmonics.A tone
tually, to explainour data would requirethat the relevant
of brief durationlimits the amountof &phasing that can
aspectof the excitationpatternshouldbroadenalongthe occur,andthislimitationisgreatestif themistunedharmon-
tonotopiccoordinate astheleveldecreases.
Thisparadoxical ic hasa lowfrequency. For example,if thesecond harmonic
resultisan exampleof theconsiderable
difficultyinvolvedin of 200 Hz is mistunedby 2%, then that harmonicis de-
developinga comprehensive model for the segregation of phasedat an 8-Hz rate,for a total of 144degafter 50 ms.Hit
mistuned harmonics. ratesfor the firstseveralharmonicswerelow enoughto per-
mit an estimateofa dephasingthreshold.The resultwas 171
D. Effect of duration degwith a standarddeviationof 54 deg,with no particular
The dependence of performanceon tonedurationcan dependenceon level. Apparently, the auditory systemcan
givecluesto the process involvedin segregating
a mistuned work with rather small valuesof the dephasing,approxi-
harmonic.If, for instance,the processwereidenticalto fre- matelyequalto half a cycle.
quencydiscrimination, thenonewouldnotexpecta signifi- Although a dephasingthresholdof half a cyclehasan
cantdurationdependence, solongasdurationsare longer intuitiveappeal,it isnotclearthattheconceptofa dephasing
than about50 ms.This is sobecausefrequencydifference thresholdis very useful.The data of Moore et al. (1986) for
limensat intermediatefrequencies do not changemuchas 410 and 50 msdo not showa particularlylargeeffectat low
tonedurationis increasedbeyond50 ms. harmonic numbers.For all the listeners,the thresholdsfor
thesetwo durationsare roughly parallel as the harmonic
numberisincreased.To explainthisresult,onemightinvoke
I. Hit rates and thresholds two limitationsimposedby shortdurations,a dephasing li-
The two experimentsby Moore et al. (1985) usedfour mitationfor low-frequencyharmonicsand a spectralresolu-
durations: 50, 110, 410, and 1610 ms. The different dura- tion limitation for higherharmonics.
tionsledto differentfunctionswhenthethresholdmistuning
was plotted against the mistuned harmonic number, n.
2. The perceived delay of the mistuned harmonic
When the cues were unrestricted, the thresholds for 410 and
1610 ms decreasedwith increasingn. This behavioris ex- A peculiareffectoccurswhena mistunedharmonicex-
pectedif thedetectionof inharmonicityis mediatedby beats perimentis run at short durationssuchas 50 ms. Listeners
or roughnessandif thekeyelementin detectionis therate of hear the mistunedharmonicsegregatedfrom the complex
this kind of modulation. A constantrate, and henceconstant tone,but the mistunedharmonicemergesfrom the complex
sensationaccordingto this explanation,correspondsto a tone only after a delay. The effectis striking. It was noticed
decreasingpercentagethreshold(decreasingas I/n) as n by the threelistenersin this study.It wasnoticedby all the
increases.If this explanationis correctthen one would ex- listenersin Hartmann's 1985study.Listenerscannotagree
pectthat, for tonedurationsas shortas 50 ms, whereit is whetherthe mistunedharmonicpersistsafterthe endof the
hardto detectbeatsor roughness,the thresholdsshouldnot complextoneor whetherit endstogetherwith the complex
decreaseas a functionof n. The data from the 1985 study tone.They are all agreedthat the mistunedharmonicseems
agreeonthispoint.For 50-mstonestherewasno systematic to start later than the complextone.
tendencyfor thresholdsto decreaseasthe mistunedharmon- ßThe perceiveddelay is consistentwith a two-part idea
ic number increased. aboutauditory organization:The first part is that segrega-
One further expectsthat, if the listenersin the 1986 tion of a tonefrom a complexrequiresinformationand that
studyby Moore et al. were successful in ignoringbeatsor integrationis the defaultoperationin the absenceof infor-
roughness,as they were instructedto do, then the tendency mation.The secondpart is that informationaboutmistuning
for thresholdsto decreasewith increasingn shoulddisap- is acquiredovertime, possiblyas the mistunedharmonicis
pear,evenfor durationsof 410 and 1610ms. This too was dephasedwith respectto neighboringharmonics.A corre-
borneout by the data, as shownin Fig. 2 of that article. spondingtwo-partideain the contextof streamsegregation
Our own experimentson durationdependence useda wasproposedby Bregman(1978).
fundamentalfrequencyof 200 Hz, and levelsof 60 dB per
componentand 30 dB per component.There were two dura- E. Segregation versus resolution
tions, 50 and 410 ms. These conditions are a subsetof the The ability of listenersto hear out a harmonicin a peri-
conditionsusedby Mooreet at., permittinga directcompari- odic complextone was studiedin experimentsby Plomp
son.ListenersB and $ did I0 runswith mistunings of I%, ( 1976pp. 2ff;seealsoPlomp, 1964andPlompandMimpen,
2%, and 4%. 1968). The experimentaltechniquewas straightforward:

1719 d. Acoust.Soc. Am., Vol. 88, No. 4, October 1990 Hartmannot al.: Mistunedharmonic 1719
The listenerhearda complextonewith a dozenharmonics.
Then, the listenerheardtwo sinetones;onesinetonehad a
frequencyequalto oneof the harmonics,the othersinetone
had a frequencythat washalfwaybetweentwo harmonics.
Thelistener's taskwastochoose whichsinetonewaspresent
as a harmonicin the complextone.The data showedthat 4o
listenerscouldeasilyhearout harmonieswith low harmonic o

numbersbut experienced increasingdifficultyas the har-

o -_ o •o
monic number of the sine probe increased.The harmonic o o - •
numberfor whichlisteners made75% correctresponses was
takento bethethresholdfor resolution. Suchanexperiment o ø o •8
measures theresolvingpowerof theauditorysystem,treated o o
as a spectrumanalyzer.The fact that the resolvingpower o o o o
closelyparallelsthe criticalbandwidth, measuredby other
methods,suggests that the resolvingpoweris essentially a
1. 0
! 0o0 OOoõ
0
measureof auditoryfilter bandwidth. o o
Resolvingpowerisquitedifferentconceptually fromthe
powerto segregate a mistunedharmonic.The dataarediffer- Oo
0 ,
ent too.Perhapsthe mostdramaticillustrationof the differ- Miss assignment
enceis that Plompfoundthat the resultsof his resolving-
power experimentswere unchangedwhen the harmonic
complextonewasreplacedby a complexhavinginharmonic FIG. 10. Each circle representsan averagestandarddeviation (s.d.) of
partials.By contrast,ourexperiments onsegregation depend missesassigned to the harmonicnumber(n) givenon the horizontalaxis.
The s.d. has beendividedby n and multipliedby 100 to put the plotted
uponinharmonicityasthemajorcue.It wouldbeimpossible valueson a percentbasis.The solidlineshowstheexpectedresultfor ran-
for listenersto do our experimentin the caseof inharmonic dom matching.The experimentals.d.fallsappreciablybelowthe solidline
complextones.Unlike resolutionexperiments, it is not at all whenlisteners canhearouta harmonicthatisnotmistuned(topofgraphat
evident that narrow filter bandwidths are a benefit to the 6% = 1 semitone).

listenerin a segregationexperiment.
Curiously,it is possibleto gain informationaboutaudi-
tory frequency resolution from our mistuned harmonic
matchingexperiments.But the informationdoesnot come
from the hit rate; the information comes from the misses.
Figure 3 clearlyshowsthat, whena listenermakesincorrect
matchesneara low harmonic(n = 1-5) the matchingfre-
quenciesare not randomlyscattered;instead,they tend to
lineupatcorrect harmonic values.Thisshows thatlistener• relativeterms,dividing all thes.d.'sby theassignedharmon-
arehearingouttheharmonicthattheyarematching,evenif
it is the wrong(i.e., not mistuned)harmonic.This ideacan
bemadequantitativeby measuringthewidthof thescatterof
matchesaround harmonic values. If the width is small, then
that is evidence that the listener heard out the harmonic as a
resolvedcomponent.
To do the scatteranalysis,we first assignedeachmissto
a bin, labeled by the harmonic number closestto the miss.
For instance, matches at harmonic numbers 4.3 or 3.6 were
assigned
to thebinfor the4th harmonic.A separate
analysis
wasdonefor eachlistenerandeachexperimental condition,
i.e., for each block often matches.If there were lessthan four
matchesin a bin, then that bin was dropped.If there were
four or more matches in a bin, then the unbiased standard
deviationof the matcheswascomputed,aboutthe (possibly
nonintegral)mean.
A convenient reference is the value of the standard devi-
ation for randommatching.For example,a randommatch
in the 4th harmonic bin is equally likely to be anywhere
between 3.5 and 4.5 harmonic numbers. The value of the
standarddeviationfor randommatchesis thusequalto the
squarerootof theintegralofx 2between-- 1/2 and + 1/2,
i.e., the squareroot of 1/12, namely0.289. The data showed
that s.d.'s were much less than this value for matches to
harmonics1-4, but a few s.d.'sapproachedthis value for
harmonic5. The datacanbedisplayedmoreconveniently in
ic number.A scatterplotshowingall thedataisgivenin Fig.
10.For reference,the top of the graphisat 6%, equivalentto
a musical semitone.4 Also shown is a curve that is the line
0.289/n for randommatching.
The collection of dataon themisses was,by nature,op-
portunisticand not systematic.Nonetheless,the data do in-
dicatea dramaticchangeat aboutthe 5th harmonic.This is
in nearagreementwiththeresultsof PlompandMirapenfor
hearingout the harmonics of a 200-Hz fundamental.
They
actuallyconcluded that the 5th wasthe highestharmonic
that could be resolved.
But althoughour data can be used,as above,to learn
aboutfrequency
resolution,
theprincipalpurpose
of thissec-
tion of the paperis to emphasizethe considerable difference
betweenresolutionandsegregation. In particular,thereisno
contradiction
whenPlompandMirapenfindthat thehigh-
est resolvable harmonic of 200 Hz is the 5th and when we
findthatlisteners
cansuccessfully
segregate
a 10thharmon-
ic that hasbeenmistunedby only 1%.
IlL PITCH SHIFTS
As notedabove,thematchingdatarevealed
pitchshifts.
The pitch of a sinetone (the mistunedharmonic) embedded

1720 J. Acoust. Soc. Am., VoL 88, No. 4, October 1990 Hartmann eta/.: Mistuned harmonic 1720
 lO.

8. 8.

6. 6.

4.

2. 2.

o.-- O.

•2o •2.

-•. -4.

•6.

-10.0 2 4 6 8 10 12 14 16 -10.0 ' 4 6 8 10 12 14 16

MISTUNED HARMONIC MISTUNœD HARMONIC

FIG. 11.Pitchshiftofa mistuned harmonicasa percentage

ofthefrequency FIG. 12.SameasFig. I I but for listenerS.
ofthemistuned harmonic. forlistenerB.Fundamentalfrequencyof200Hz,
levelof 60 dB/componentand durationof 410 ms.Opensymbolsare for
positive
mistunings; filledsymbols
arefornegativemistunings.
Theamount
of mistuning isa parameter:Circlesfor 0.:5%mistuning,
squares
for 1%,
trianglesfor 2%, diamondsfor 4%. Somesymbolshavebeenslightlydis-
placedhorizontallyfor clarity.

8.

in a complextoneisdifferentfromthe.pitchin isolation.As a 6.
result,the frequencyof the isolatedmatchingtonethat best
matchesthe pitch of the mistunedharmonicis not equal to
4.
the frequencyof the mistunedharmonic;it is shifted.There
havebeenpreviouseffortsto measurethe pitchesof thecom-
2.
ponentsof a complextone (Terhardt, 1971;Houtsma,198!;
Petersetal., 1983).The mistunedharmonictechnique isone
way to makeharmoniesstandout from the complexsothat O.

theirpitchescanbematched.
The data to be presentedhere were takenfrom those •2.
pitch matchesthat were foundto be "hits." Therefore,for
any mistuned harmonic, the standard deviation of the
matcheswas within the 2.5% self~consistency window. We
further requiredthat at leastfour matchessurvivethe self-
consistency testto calculatea meanpitch shift. Becauseof
theserequirements, it wasnot possible
to finda pitchshiftfor
every mistunedharmonic number in every condition. The
shift itself was calculatedas a percentagedeviation of the
matchingfrequencyfrom the actual frequencyof the mis- 10o 2 4 6 8 10 12 14 16
tuned harmonic. Results for listenersB, S, and K are shown MISTUNED HARMONIC
in Figs.11-13,respectively,
for thecaseofa 200-Hz funda-
mental, 60 dB per componentand 410-msduration. Open
symbolsin the figuresshowshiftsfor runswith positivemis- FIG. 13.Sameas Fig. I l but for lislenerK.

1721 J. Acoust.Soc. Am., Vol. 88, No. 4, October 1990 Hartmann eta/.: Mistuned harmonic 1721
tunings;closedsymbolsare for runswith negativemistun- closedsymbolscorresponding to negativemistuning.For
ings.On runswith negativemistuning,only the 12 lowest 65% of thosepoints,the pitchshiftis largerthan the stan-
harmonicswerecandidatesfor mistuning. darddeviation.This is evidencethat the negativepitchshift
The figuresshowthat positivemistuningsusuallyled to is genuine.
positivepitchshifts;thistendency isverystrong.The figures Althoughthe Terhardtalgorithmfor pitchappearsto
alsoshowthat negativemistunings tendedto producenega- fail, it still is quitepossiblethat the pitch shiftsobservedin
tive pitchshifts.The effectfor negativemistuningsis not so ourexperiment resultfromeffectsthataretonotopically lo-
strongnorconsistent astheshiftforpositivemistunings, but cal.If thisisso,thenthepitchshiftsshouldbesensitive to the
it is clearlypresentin the data.In otherterms,mostof the separation betweenadjacentharmonics, or spectraldensity.
datashowa formof"overshoot"effect.Whenthemistuning To test this idea, we studiedthe pitch shiftsobtainedfor
is positive,the pitch is shiftedto a still morepositivevalue, nominalfundamental frequencies of 400 and 800Hz. Only
and,whenthemistuningisnegative,thepitchbecomes even positivemistuningswere used.The amountof significant
more negative. datawaslimited;pitchshiftswerenormallygreaterthanthe
The pitchmodelof Terhardt (1979) andhiscolleagues standard deviations for harmonies 3, 5, 6, and 7 for 400 Hz
(Terhardt et aL, 1982) includesan algorithmfor the calcula- but only for the third harmonic for 800 Hz. Becausethe
tion of the spectralpitchesof the harmonics.The algorithm spectraldensityis smaller for an 800-Hz fundamentalthan
has the flavor of a place theory and doesnot requirethe for 400 Hz, oneexpectssmallerpitchshiftsfor the 800-Hz
partialsof a tone to be strictly harmonic.It is, therefore, fundamental in anygivenfrequencyregion.Thisexpectation
applicableto our stimuli.Accordingto the algorithm,the wasborneout by thedata:The shiftfor the third harmonicof
spectralpitchesare shiftedbecauseof partial maskingef- 800 Hz was 1.8%, considerablylessthan the shift for the
fects.For example,thespectralpitchof thefundamentalof a sixth harmonic of 400 Hz, which was 4.8%.
complextoneisshifteddownwardbecause theexcitationfor Similarly,onemight expectthe pitchshift to decrease
the fundamentalis maskedfrom aboveby higherharmonics, with decreasingsignallevel becausea smallersignallevel
mainlythe second.The predictionsof the algorithm,calcu- should lead to less interaction between the mistuned har-
lated from the formulasgivenin Terhardt et al. (1982), are monicand its neighbors.To testthis idea, the three listeners
shownin Fig. 14. The figureshowspredictionsfor -- 4% did runswith a 200-Hz fundamental,a mistuningof 2%, and
and 4% mistuning;the predictionsfor other valuesof mis-
tuning actually used in our experimentslie between the
curvesfor -- 4% and4%. The algorithmpredictsa shiftfor
zero mistuning(harmoniccomplex), approximatelymid-
way betweencurvesfor -- 4% and 4%, althoughPeterset
aL (1983) did not find such shifts.
Comparingthe predictedshiftswith the observedshifts
showsthat the algorithmcorrectlypredictsthe trendof the
data when the mistuningis positive.When the mistuningis
negative,however,the algorithmfails completely.Experi-
mentally,negativemistuningusuallyleadsto negativepitch
shifts. By contrast,the algorithm predictspositivepitch
shiftsfor negativemistuning.Figure14showsthat, for har-
monics2-7, the predictedpitchshiftsfor negativemistun-
ingsareevenmorepositivethanthepredictedpitchshiftsfor
positivemistunings. This is a resultof theupwardspreadof
masking.For the third harmonic, for example,the major
contributionto the pitch shiftisthe secondharmonic.When
the third harmonicis mistunednegatively,it comescloserto
the secondharmonic,and sothe pitch shiftbecomeslarger
(morepositive).
• The factthatthealgorithmfailssobadly
for negativemistuningsuggests that thereis somethingquite
wrongwith the ideathat pitch shiftsare mainly determined
by partial masking.
The reliabilityof pitchshiftdatacanbeassessed by com- I I I I [ I I I I I I I I I I I

paring the standarddeviationswith the pitch shiftsthem- -10.0 2 4 6 8 10 12 14 16

selves. The standard deviations were calculated for individ- MISTUNED HARMONIC
ual listeners across the matches of different runs. The
separatematchesthat contributed to a standard deviation
werethosethat satisfied thewindowingcriterionfor accept- FIG. 14.Predicted
pitchshiftfromthealgorithmofTerhardtetaL (1982)
ing data, thus,a minimumof four matchesand a maximum fora fundamental
frequency of200Hz anda levelof60dBpercomponent.
The opendiamonds are for 4% mistuning;the closeddiamonds arefor
of ten. Our mostcompletedata are for nominalfundamen- -- 4% mistuning,
andthesolidlineisfornomistuning. If thealgorithm
is
tals of 200 Hz. Here, the figuresfor B, S, and K have 102 correctthenthesepredicted
shiftsshouldagreewiththedatain Figs.I I -I 3.

1722 J. Acoust. Soc. Am., Vol. 88, No. 4, October 1990 Hartmann eta/.: Mistuned harmonic 1722
a durationof 410 ms,usingthreedifferentsignallevels.Per
component, theselevelswere60, 30, and 12 dB SPL. The
goaloftheexperiment wastocompare pitchshiftsfora given
mistuned harmonic across the different levels. The matches
thatpassed theacceptance testspermitted36comparisons of
60 dB with 30dB,and 15comparisons of 30dB with 12dB.
Of these51comparisons therewereonlynineeases thatwere
inconsistentwith the rule that lower signallevelslead to
smallerpitchshifts.
IV. CONCLUSIONS
The abilityof a listenerto segregate
a mistunedharmon-
ie from a complextonebackgroundis an exampleof audi-
tory organizationbaseduponsteady-state cues.We began
ourstudyof thiseffectwiththeintentionof improvingonthe
dataof Mooreandhiscolleagues, byusinga matchingexper-
imentinsteadof a discrimination experiment.We foundthat
thematchingexperiment hasa numberof advantages, but it
is not perfect.First, we wererequiredto discardasartificial
successful matches for harmonies above the 12th. We attrib-
uted theseto localized roughnessor beats and not to the
segregation of a mistunedharmonic.Thisproblemoccurred
for a fundamentalfrequencyof 200 Hz; it did not occurfor
fundamentals of 400 and 800 Hz. Second,usingthe match-
ing techniquedid not eliminatethe largeintersubjectvari-
ability that seemsto be characteristicof mistunedharmonic
detectionexperiments.
The matchingtechnique enabledusto extendmeasure-
mentsto higherfrequencies, andthisrevealedan important
absolutefrequencylimit, between2.2 and 3.5 kHz, for the
segregation ofa mistunedharmonic.We understandthisre-
sult by supposing that segregation
is mediatedby the detec-
tion of asynchronyin neuralfibersstimulatedby both the
mistunedharmonicand neighboring harmonies.The high-
frequency barrierthenreflectsa limitonencoding byneural
synchrony asfrequency increases.
For frequencies below2 kHz, segregation thresholds,
expressed as a percentof mistunedharmonicnumber,ap-
pearedto be roughly independentof mistunedhartsonic
number,neglectingspecialeffectsfor the fundamentaland
individual differences. To some extent this result reflects our
dataanalysismethod.Our criteriafor a "hit" includeda self-
consistency
window that was a constantpercentageof the
mistuned harmonic number. Had we chosen a window with
a constantfrequencywidth for all mistunedharmonicsthen
segregation thresholds,expressed as a percentof mistuned
harmonicnumber,wouldhaveincreased for increasing
mis-
tunedharmonicnumber.However,thismethodological ar-
bitrarinessis unlikelyto havehadan importanteffecton our
conclusions because,when detectionperformancedeterio-
ratesat high frequency,it deterioratesdramatically.With
eitherwindowcriterion,the resultsshowdearly that segre-
gationthresholdsdo not decreaseasthe mistunedharmonic
number increases.By contrast, earlier thresholds from
Moore et al. (1983), where detectioncould be basedupon
any cue,did decreasewith increasingharmonicnumber.
Further experimentsfound that segregationperfor-
manceshowedparadoxicalbehaviorasa functionof overall
signallevel. A mistuned7th harmonic was more easilyde-
723 d. Acoust.Soc. Am.,Vol. 88, No. 4, October 1990
teetedat 60 than at 30 dB, but a mistuned4th harmonic was
moreeasilydetectedat 30 thanat 60 dB. If thedetectionof a
mistunedharmonicrequiresan overlapbetweenexcitation
patternsassociatedwith the mistunedharmonic and its
neighbor (s), and if excitationpatternwidthsincreasewith
increasing level,thentheleveleffect,if any,shouldbeoppo-
siteto the onethat we observed.Experimentswith tonesof
briefdurationshowedthat theamountof waveformdephas-
ing requiredto segregate a harmoniccanbe small.
An interestingvirtueof the matchingmethodis that the
data usedto determinesegregationperformancecan alsobe
usedto studya quitedifferentmatter, namely,the pitchesof
thespectralcomponents of a complextone.Our experiments
foundthat there are pitch shiftsand that thesetend to be
exaggerated versionsof the mistuning;i.e., positivemistun-
ingleadsto positivepitchshiftandnegativemistuningleads
to negativepitch shift.The negativeshiftsare smallerand
lessstablethan the positiveshifts,but they are undeniably
real.By contrast,a modelwherebypitchshiftsarecausedby
partial maskingof excitationstrengthpredictsthat pitch
shiftsshouldbe positivefor bothpositiveand negativemis-
tunings.
ACKNOWLEDGMENTS
The workof WMH wassupported bytheNationalInsti-
tutesof Health and by the US-France CooperativeScience
Programof the National ScienceFoundationand the Centre
National de la RechercheScientifique.
APPENDIX: EXPERIMENT DETAILS
This Appendixincludesfurther detailson the experi-
mentalproceduresand the analysis.
1. Equal,zed stimuli
Mostof thepitchmatching
experiments,
whichprovide
data for this article, were done twice. In the first set, the
electricalsignalsentto the phoneshad 16 harmonicswith
equalamplitudes.For the secondset,the harmonicampli-
tudesof theelectricalsignalwereadjustedsothattheampli-
tudesof theacoustical signal,asmeasured witha spectrum
analyzerandflat-platecoupler,wereall equal.Eachsetcom-
prisedten matchesby a listenerto eachmistunedharmonic
for each percentageof mistuning.Although some data
pointsfor individuallistenersweredifferentfor thetwo stim-
ulussets,therewaslittle agreement amongthedatafor dif-
ferentlisteners with respectto potentialeffects of equaliza-
tion.Because therewasnosystematic effectof equalization,
we combined the data for the two stimulus sets. Data from a
third setof tenmatches,
with 16unequalized harmonics of
whichonlythefirst12mightbemistuned, werealsoaddedto
the setwhenit wasdiscoveredthat theydid not differin any
importantway from the dataof the othertwo sets.Increas-
ingtheamountof datain thiswayhadtheeffectof smooth-
ing the data for individuallisteners,a desiredresultfor our
purposes.
Hartmanneta/.: Mistunedharmonic 1723
2. Success by persistence grationofauditoryentities,"inAuditoryFunction,
editedbyG. M. Edel-
man,W. E. Gall, and W. M. Cowan (Wiley, New York), pp. 623-645.
In a matchingexperimentof our kind, misleadingre- Houtsma,A. J. M. (1981). "Noise-inducedshiftsin the pitch of pure and
suitsmaybeobtainedifs listenerpersistentlymakesa partic- complextones,"J. Acoust.Soc.Am. 70, 1661-1668.
Johnson,D. H. (1974). "The response
of singleauditory-nervefibersin the
ular match. If, for example, the listener always matchesthe
cat to singletones;synchronyand averagerate," Ph.D. thesis,MIT,
third harmonicof thecomplextone,then,on the trialswhere Cambridge,MA.
the third harmonic is actually mistuned,the listenerwill Klein, M. A., and Hartmann W. M. (1981). "Binaural edgepitch," 1.
score100% correct,evenif he cannotidentify the third har- Aconst. Soc. Am. 70, 51-61.
Marin, C. M. H., McAdams, S., and Lienard, J-S. (1989). "The role of
monicasthecorrectanswer.Thiseffectisa potentialsource beatsin concurrentsoundsegregation basedon frequencymodulation
of error. We discoveredonly one caseof such persistent cues,"in Proceedings of the 5th Annualmeetingof theInternationalSo-
matching,listenerK at 200 Hz, 60 dB/eomponentand0.5% cietyfor Psychophysics, Ca*sis,France,pp. 120-125.
mistuning.Here, it wasnecessary to correctthe hit rate for MeAdams,S. (1984a). "Spectralfusion,spectralparsingandtheformation
of auditoryimages,"Ph.D. thesis,StanfordUniversity.
the mistunedthird by subtractingthe rather large percen- MeAdams,S. (1984b). "The auditoryimage:A metaphorfor musicaland
tageof matchesto the mistunedthird harmonicwhen har- psychological researchon auditoryorganization,"in Cognitioe Processes
monics 1,2,4, and 5 were mistuned. in the Perception of.4rt, editedby R. Crozierand A. Chapman(North
Holland, Amsterdam), pp. 183-187.
MeAdams,S. (1989). "Segregation of concurrentsounds.I: Effectsof fre-
To testouranalysis
method,
wereversed
theorderofoperations:
Wefirst quencymodulationcoherence,"J'.Acoust.Soc.Am. 86, 2148-2159.
foundthehit-rateplotsfor eachpercentage of mistuning,
averaged over Moore, B.C. J. ( 1982). An Introductionto thePsychologyof Hearing ( Aca-
the threelisteners,
givenasFig. 4, andtheninterpolatedfor the 60% hit demic, London), 2nd ed.
rateto finda singlethreshold
plot.That plotwascompared withtheplot Moore,B.C. J., Peters,R. W., andGlasberg,B. R. (1985). "Thresholdsfor
shownin Fig. 5. For eachof the 16mistunedharmonics,thedifference
was thedetection ofinharmonicity in complextones,"J. Acoust.Soc.Am. 77,
nevergreaterthan 0.3 percentage points,exceptfor the mistuned7th, 1861-1867.
wherethe differencewas0.4 points. Moore, B.C. J., Peters,R. W., and Ginsberg,B. R. (1986). "Thresholdsfor
Thatlisteners
canidentify
frequency
ranges
wherebeating
takesplaceina hearingmistunedpartialsas separatetonesin harmoniccomplexes,"J.
broadbandcontextwas recentlyshownby Marin et al. (1989). Acoust. Soc. Am. 80, 479-483.
Theimprovement
indetecting
some
mistuned
harmonics
asthesignal
lev- Moore,B.C. J., andGlasberg,B. R. (1989). "Differencelimensfor phasein
el is decreaseddid not continueindefinitely.Runs done at a level of 15 normal and hearing-impairedlisteners,".I. Acoust.Soc.Am. 86, 1351-
dB/componcnt showedhit ratesthat were alwayslessthan the ratesfor 60 1365.
dB/component. Moorer, J. A., Chauveau,A., Abbolt, C., Eastty,P., and Lawson,1. (1979).
4Data for the s.d. of the missescome from ten-run blocks as follows: five "The 4C machine,"Cornput.Music J. 3, 16-24.
unequalized
at0.5% mistuning;fiveunequalizedandtwoequalized at 1%; Peters,
R. W.,Moore,B.C.J.,andGlasberg,
B.R. (1983)."Pitchofcom-
five uncqualizedand threeequalizedat 2%; five unequalizedand one ponentsof a complextone,"J. Acoust.Soc.Am. 73, 924-929.
equalizedat 4%. PIomp,R. (1964). "The earasa frequencyanalyzer,"J. Acoust.So:. Am.
5Thereisnoeasywayto explain
whythepredicted
pitchshiftbecomes 36, 1628-1636.
largerforpositivemistuning
thanfornegativewhenthemistunedharmon- PIomp,R. (1967). "Beatsof mistunedconsonances,"
J. Acoust.Soc.Am.
ic numberis greaterthan 8. Terhardt'salgorithmis semiempirical
and 42, 462-474.
owesat leastasmuchto experimental dataasto modelbuilding. PIomp,R., andMirapen,A.M. (1968). "The earasa frequencyanalyzer
II," 1. Acoust. Soc. Am. 43, 764--767.
Bregman,A. S. (1978). "Auditory streamingis cumulative,"J. Exp. Plomp,R., and Steeneken,H. J. M. (1968). "Interferencebetweentwo sim-
Psych.:Human Percep.Perform.4, 380-387. ple tones,"J. Acoust.Soc.Am. 43, 883-884.
Bregman,A. S., and Pinker,S. (1978). "Auditory streamingand the build- PIomp, R. (1976)..4spectsof Tone Sensation--/! Psychophysicai Study
ing of timbre," Can. J. Psych.32, 19-31. (Academic, London).
Gardner, R. B., and Darwin, C. 1. (1986). "Grouping of vowelharmonics Rasch, R. A. (1978}. "The perceptionof simultaneousnotesas in poly-
by frequencymodulation:Absenceof effectson phonereit categoriza- phonie music," Acustica40, 21-33.
tion," Percept.Psychophys. 40, 183-187. Terhardt,E. ( 1971}. "Pitch shiftsof harmonics,an explanationof the oc-
Hartmann,W. M. (1985). "Perceptualentitiesfromcomplexinharmonic laveenlargementphenomenon," Proc.7th ICA 3, 621-624.
tones," Assn. for Researchin Otolaryngology8th Meeting, pp. 168 Terhardt,E. (1979). "Calculatingvirtualpitch,"Hear.Res.1, 155-182.
(airs). Terhardt,E., Stoll,G., and Sweewann,M. ( 1982}. "Algorithmfor extrac-
Hartmann,W. M. (1986). "Pitchand the perceptualorganizationof com- tion of pitch and pitch saliencefrom complextonal signals,"J. Aconst.
plexspectra,"J. Acoust.So:. Am. Suppl.1 79, S65. Soc. Am. 71, 679-688.
Hartmann, W. M. (1988). "Pitch perceptionand the segregation
and inte- Whitfield, L C. (1967). The.4uditoryPathway(Arnold, London).

1724 J. Acoust. Soc. Am., Vol. 88, No. 4, October 1990 Hadmann et al.: Mistuned harmonic 1724