Perception & Psychophysics

1986, 40 (2), 69-73

Hierarchical organization of temporal patterns

PETER J. ESSENS
TNO Institute for Perception, Soesterberg, The Netherlands

In two reproduction experiments, limitations of a hierarchical organization of temporal pat-

terns were investigated by varying the way lower levels relate to higher levels in the hierarchy.
From the systematic errors subjects made in reproducing temporal patterns, it is concluded that
an accurate internal representation will be arrived at only if the temporal structure of a pattern
enables an organization in which hierarchical levels relate as integers with prime factors 2 or
3. Implications for the temporal pattern perception theory are discussed.

A general notion in characterizing the cognitive organi-
zation of temporal patterns is that the time structure is
coded in some hierarchical form (Jones, 1976, 1981b;
Longuet-Higgins & Lee, 1984; Martin, 1972; Michon,
1974). Hierarchical organization is typically pictured ab-
stractly as a tree representing multiple levels of time units
with lower levels nested within higher levels (see Jones,
1981a). Various theoretical representations to describe the
time structure of a temporal pattern differing in the way
lower levels relate to higher levels have been proposed
(Longuet-Higgins, 1978). This study is concerned with
the actual organization of higher and lower levels in a hi-
erarchical representation of temporal patterns.
Models specifying which characteristics of temporal
patterns determine their actual organization have been pro-
posed by Povel (1981, 1984) and Povel and Essens (1985).
Related models confined to temporal patterns that occur
in music have been presented by Longuet-Higgins and Lee
(1982, 1984).
Povel (1981) found empirical support for a hierarchi-
cal model of temporal pattern perception that consists of
at least two levels: a higher order level, called beat, and
a lower level of subdivisions of that beat level. This or-
ganization is arrived at by segmentation of the temporal
pattern into equal intervals, the beat unit, and expressing
the intervals smaller than the beat unit as a subdivision
of the beat unit in which they occur. Thus, a temporal
pattern with intervals of 250250250750 (msec) can be
represented, according to the model, as two beat units of
750 msec, the first unit subdivided into three equal parts.
In later studies, Povel (1984) and Povel and Essens (1985)
extended this idea, and in the latter article they introduced
the concept of internal clock, comparable to the beat
concept.
The clock model assumes that, in perceiving a temporal
The research reported here was supported by a grant from the Nether-
lands Organization for the Advancement of Pure Research (ZWO) and
was carried out at the University of Nijmegen, The Netherlands. The
author wishes to thank D. J. Povel and G. B. van der Voet for helpful
comments on the manuscript. Requests for reprints should be sent to:
Peter Essens, TNO Institute for Perception, Kampweg 5, P.O. Box 23,
3769 DE Soesterberg, The Netherlands.
pattern, a listener tries to generate an internal clock which
is subsequently used as a device to specify the temporal
structure of the pattern. Furthermore, the model assumes
that the distribution of the accents perceived in the se-
quence determines whether an internal clock actually is
induced and which clock will provide the hierarchical or-
ganization of the pattern. For example, consider the tone
pattern with intervals 600 200 400 200 200 200 600
(msec). A clock with unit 600 (msec) organizes the pat-
tern in four equal parts (600) (200 400) (200 200 200)
(600). On the other hand, when a clock unit of 800 (msec)
is used, the pattern is segmented in (600 200) (400 200
200) (200 600). The clock model specifies the strength
of induction of each theoretically possible clock. It was
shown that temporal patterns conceived with a strongly
induced clock are reproduced more accurately than pat-
terns not so conceived (Essens & Povel, 1985; Povel &
Essens, 1985). This paper is concerned with patterns that,
in terms of the clock model, have a strong clock induction.
The purpose of the present study was to determine
which subdivisions can be adequately represented in a hi-
erarchical model. In two experiments, subjects reproduced
temporal patterns in which subdivisions of the higher order
level were systematically varied.
EXPERIMENT 1
Three ways of subdividing the clock unit can be distin-
guished: empty, filled with equal intervals, and filled with
intervals that subdivide the clock unit unequally. For ex-
ample, in the temporal pattern with intervals 600 200 800
400 200 200 800 (msec) with a theoretical clock unit equal
to 800 msec, according to the clock model, the first clock
unit is subdivided into two parts with a ratio of 3: 1. The
third unit is subdivided into three intervals with a ratio
of 2: 1:1. The other units are not subdivided. To deter-
mine the nature of the relationship between clock level
and its subdivisions, other, more complex ratios must also
be studied. For example, consider the temporal pattern
700 200 900 500 200 200 900 msec. The theoretical clock
unit, equal to 900 msec, is subdivided into parts that form
nonintegers (700 200 and 500 200 200). As it is formu-
lated now, the clock model emphasizes the necessity of

69 Copyright 1986 Psychonomic Society, Inc.

70 ESSENS

Table 1
Patterns Presented in Two Conditions of Experiment 1
Noninteger Integer
Set I
1200 200 500/200 500 200/900/9001 1200 200 400/200 400 200/80018001
1500 200 200/200 500 200/900/9001 1400 200 200/200 400 200/800/8001
1700 200/200 200 5001500 200 200/9001 1600 200/200 200 4001400 200 200/8001
Set 2
1500 200 2001700 200/700 200/9001 1400 200 200/600 200/600 200/8001
1500 200 200/200 700/200 700/9001 1400 200 200/200 6001200 600/8001
1500 200 200/200 7001700 200/200 7001 1400 200 200/200 6001600 200/200 6001
1500 200 200/200 700/200 700/200 7001 1400 200 200/200 6001200 600/200 6001
Note-Intervals in milliseconds. Slant lines mark clock units. The subdivisions under considera-
tion are set in bold type. The subdivisions relate basically as 2.5:1 and 2:1 in Set 1, and 3.5:1
and 3: 1 in Set 2.

using an internal clock to arrive at an accurate represen- and practicing the stimulus involved. Stimulus generation and
tation. If the presence of an internal clock as a metric response registration were controlled by a PDP-ll/03 computer.
device is a sufficient condition for an adequate descrip- Subjects. Twenty subjects, students in psychology, participated
in the experiment. Ten were musically trained, having played an
tion of a temporal pattern, patterns containing subdivi-
instrument for at least 5 years.
sions with ratios 2.5: 1 and 3.5: 1 would be reproduced
as well as patterns with subdivisions with 2:1 and 3:1
Results
ratios.
Since an examination of the performance of individuals
Patterns containing subdivisions with integer ratios
revealed no differences attributable to musical training,
formed the integer condition; patterns containing subdi-
data from all subjects were pooled. Ratios were calcu-
visions not in integer ratios formed the noninteger condi-
lated from the relevant mean long and short intervals
tion. The performance in the two conditions was com-
reproduced in each sequence. In Figure 1, these ratios
pared using a reproduction paradigm.
are presented averaged over subjects for the patterns in
the conditions of the two sets separately. Large deviations
Method
Stimuli. Two sets of temporal patterns were formed in which
were found in the noninteger condition: reproduced ra-
the relevant ratios of the subdivisions of the clock units were 2.5: I tios, which showed a strong tendency toward integer ra-
(e.g., 500 200 2(0) or 2:1 (e.g., 400 200 2(0) in Set 1 and 3.5: I tios in both Set 1 and Set 2, differed significantly from
(e.g., 7002(0)or3:1 (e.g., 6002(0) inSet 2. In Table I, the tern- the presented interval ratios [t(I9) = I3.7,p < .01, for
poral patterns are shown as sequences of tone onset intervals (msec) Set 1, and t(I9) = 9.9, p < .01, for Set 2J. The means
with the clock units marked by slant lines and the subdivisions un-
der consideration set in bold type. The duration of the shortest in-
tervals was 200 msec in all patterns.
The tones in the stimuli were made of 50-msec 830-Hz square 3.5
, , .
waves with 5-msec rise and fall times. The tones were presented
"tJ
set2
to the subjects in combination with a low-pitch isochronic tone se- 01 non- integer
u
quence, whose only purpose was to strongly induce the indicated :J ..... .........
00-_ - -0-- - -~- ...... .....
3 ........ -... .,. _e.... .... ........--0
clock. This isochronic sequence consisted of tones of 50-rnsec "tJ
125-Hz square waves with 5-msec rise and fall times. For exam- 0
L.
ple, in the temporal sequence consisting of the time intervals 200 o,
Ol
integer •

-
200 500 200 500 200 900 900 (msec) with a theoretical clock unit L.

equal to 900 (msec), the low-pitch tones coincide with the first, .Q
fourth, seventh, and eighth tone of the sequence.
2.5
Procedure. The experiment was performed in two parts, first
0
L. set1 non-integer
the noninteger condition and then, after a short pause, the integer
.....
0 ~~__~ - -0- - - -_~_:..--~

-
> - -e
condition. Order of presentation of the stimuli within each part was L. - -e- - - integer .
randomized. The task was practiced before each part of the experi- 01 2
ment. The stimulus was presented cyclically through loudspeakers. c
The subject's task was to reproduce the presented pattern as pre-
cisely as possible. The subject listened to each stimulus and prac-
ticed tapping with his/her index finger for as long as he/she wished. •
The subject stopped stimulus presentation by pushing a button, and 2 3 4
then reproduced the pattern four times, cyclically, on a response pattern
plate. Each tap produced the same tone as that used in the stimu-
lus. If variability of performance of a pattern was such that stan- Figure 1. Mean reproduced interval ratios for the noninteger and
dard deviation exceeded 15 % of the mean, the subject was asked integer conditions in the two sets. The dotted lines connect data points
to produce that pattern again, repeating the procedure of listening of each condition.
 HIERARCHICAL ORGANIZATION OF TEMPORAL PATTERNS
(and standard deviations) of the reproduced ratios for the
integer and noninteger conditions were 2.10 (.12) and 2.14
(.12), respectively, for Set 1 and 2.94 (.18) and 3.07
(.20), respectively, for Set 2.
Although the ratios of the long and short intervals in
the noninteger ratio condition changed in reproduction,
the sum of these intervals, making up the clock unit, re-
mained constant (r test, p > .05). In this condition
(noninteger), means (and standard deviations) for the long
and short intervals were 460 (18.4) and 216 (8.6) for Set 1
and 667 (37.9) and 217 (10.1) for Set 2, respectively. In
the integer ratio condition, means (and SDs) were 417
(18.7) and 199 (6.1) for Set 1 and 594 (33.7) and 202
(7.0) for Set 2. Deviations of the short interval in the
noninteger condition were significant as compared with
the integer condition [F(1,19) = 196.0, P < .0001].
Discussion
Data clearly show that integer interval ratios are better
reproduced than noninteger ratios. Moreover, in
reproduction, noninteger ratios tend to be reproduced as
integers, whereas the clock unit remains unchanged. It
can be concluded that adequate representation will be ar-
rived at only if subdivisions of a present clock form in-
teger ratios. This suggests a representation in which the
clock interval is partitioned into equal parts such that the
smallest part equals the shortest interval. Intervals longer
than the smallest part can be specified by combining parts.
Subdivision of the clock unit may be characterized by
the way the intervals relate to the clock unit. In the sam-
ple pattern 600 200 800 400 200 200 800 with clock unit
equal to 800 msec, the first unit (600 200) and third unit
(400 200 200) both contain a 4-division. Experiment 2
examines the question whether a division of one clock unit
is related to the division of another clock unit in the same
pattern.
EXPERIMENT 2
In Experiment 1, the ratios of the intervals within a
clock unit were studied. To determine the hierarchical na-
ture of the temporal organization, Experiment 2 was con-
cerned with the relationship between the lower and higher
levels (the clock unit). For example, in the pattern of 200
200 200 600 400 200 600 msec with a theoretical clock
unit equal to 600 msec, the subdivisions of the first clock
unit (200 200 200) and the third unit (400 200) both re-
late to the clock unit as a 3-division; the intervals are,
respectively, Va Va and 2Ia Va of the clock unit. However,
in the pattern 300 300 600 400 200 600 msec with a clock
unit equal to 600 msec, relations between subdivision and
clock level differ; the intervals within the first and third
unit are, respectively, 1/2 V2 (a 2-division) and 2Ia Va (a
3-division) of the clock unit. The question addressed was
whether different types of division (e.g., 2-division and
3-division) were conceivable within the same pattern.
This experiment compared the reproduction of patterns
in two conditions: one in which subdivisions differed be-
71
tween clock units within a pattern, for example, 2-division
versus 5-division (different condition) and one in which
subdivisions were similar (same condition).
Method
Stimuli. Five sets of two patterns each were formed in which
subdivisions of the clock units were either different (first pattern)
or same (second pattern) (see Table 2). The clock unit in each pat-
tern was equal to the longest interval in the pattern. Sets I and 2
and Sets 3 and 4 differed only in tempo. Because the longest clock
unit used was 1,000 msec and the shortest was 200 msec, Set 5 had
no equivalent in a different tempo. Patterns in the different condi-
tion contained a 2-division in the first clock unit and a different
division in the third clock unit, whereas in the same condition, the
divisions were of the same type in the first and third clock units.
The tones that marked the intervals in the stimuli consisted of 50-
rnsec 830-Hz square waves that included 5-msec rise and fall times.
Procedure. The order of presentation of the five sets was ran-
domized; order within sets was fixed-first different, then same di-
vision. To avoid interference from other sets, each set was preceded
by a practice pattern in which the first clock unit was undivided
and the third clock unit was subdivided similar to the subdivision
in the set. For example, the practice pattern in Set I was 800 800
533 267 800 msec.
The subject's task was to reproduce the cyclically presented pat-
terns as precisely as possible. Stimulus presentation and response
procedures were similar to those in Experiment I.
Subjects. Twelve subjects, students in psychology, participated
in the experiment. Six had played a musical instrument for at least
5 years.
Results
As in Experiment 1, the data of musically trained and
untrained subjects were pooled because no systematic
differences related to musical training were found. Since
systematic effects were revealed only in the ratios of the
reproduced intervals within the third clock unit, Table 3
presents these ratios averaged over subjects and their stan-
dard deviations. Note that ratios 2:1,3:1, and 4:1 refer
to division types 3, 4, and 5, respectively, and that in the
different division condition a 2-division preceded these
division types. Figure 2 shows, for all conditions on a
scale of ratios, the relation between the ratio in the stimu-
lus (vertical line) and the ratio in the reproduced sequence
(arrow end point). The data showed strong effects of
different versus same conditions in some sets. Effects of
Table 2
Patterns in Two Conditions Presented in Experiment 2
and Division of First and Third Clock Unit of Each Pattern
Set Patterns Division
1. Different /400 400/800/533 267/800/ 2 3
Same /267 267 267/800/533 267/800/ 3 3
2. Different /500 500/1000/667 333/1000/ 2 3
Same /333 333 333/1000/667 333/10001 3 3
3. Different 1400 400/800/600 200/8001 2 4
Same 1200 200 200 200/8001600 200/8001 4 4
4. Different 1500 500/10001750 250/10001 2 4
Same 1250250250 250/10001750 250/10001 4 4
5. Different 1500 500110001800 200/10001 2 5
Same 1200 200 200 200 200/1000/800 200/1000/ 5 5
Note-Intervals in milliseconds. Slant lines mark clock units.
72 ESSENS

condition (different vs. same) were significant in Set 1 and different division
Set 2 [F(2,18) = 29.1,p < .0001, andF(2,18) = 29.1,
P < .0001, respectively]. The trend of the effect in the
conditions of Set 1 and Set 2 is toward a 3: 1 ratio. No
• I
significant differences between conditions were found in 4 ratio
Sets 3, 4, and 5. In Set 5, reproduction deviated strongly
in both conditions, suggesting that the tendency toward
the 3: 1 ratio in the different condition might be due to same division
factors other than difference in division. In agreement with
the results in Experiment 1, the observed deviations of
the interval ratios in the different conditions did not modify
the clock unit. Tempo (Set 1 vs. Set 2 and Set 3 vs. Set 4 ~-}----'--
2
• I
3 4 ratio
in both conditions) had no significant effect on
reproduction. interval ratios reproduced
Subjects had great problems with the patterns in Set 5.
Figure 2. Mean reproduced interval ratios for the different and
Verbal comments about the same condition indicated that same conditions of Experiment 2, represented on a continuum of
the six tones in the beginning of the pattern were particu- ratios as deviations from the presented interval ratio. The arrows
larly difficult to organize. refer to the stimuli. Vertical lines on the continuum mark the ratio
in the stimulus; the end points of the arrows indicate the actual
Discussion reproduced ratio.
The results may be summarized in terms of division:
(1) A 3-division preceded by a 2-division in the same pat- Experiment 1, however, 3: 1 ratios were reproduced as
tern is not reproduced accurately, whereas a 3-division smaller than 3. These differences remain to be explained.
preceded by a 3-division is. (2) A 4-division preceded by
a 2-division is reproduced accurately, as is a 4-division GENERAL DISCUSSION
preceded by a 4-division. (3) 5-divisions are reproduced
badly irrespective of the preceding division. However, The main conclusion drawn from the experiments in
deviant reproductions did not affect accuracy of the clock this study is that temporal patterns, conceivable with a
unit, from which it may be concluded that the clock unit clock unit, are represented hierarchically, with higher
is used as a reference in the representation of temporal order levels relating to lower levels as integers with prime
patterns. factors 2 and 3. Accurate reproduction was found when
Data suggest that 2- and 4-divisions are related divi- the clock unit was subdivided into parts that formed ra-
sions. This adds to the idea that in a hierarchical represen- tios that were simple integers. If parts did not form in-
tation higher levels are partitioned equally into lower teger ratios, reproduction was inaccurate, showing sys-
levels. When a 2- and a 4-division occur in one pattern, tematic errors toward integer ratios. An additional
a hierarchy with two levels below the clock unit could limitation of the subdivision of the clock unit was found
adequately describe the pattern. When a 2- and a 3- in the fact that subdivisions in a temporal pattern have
division occur in one pattern, the level below the clock to be of one kind. The fmding that 2- and 4-divisions did
unit related to the 2-division cannot be equally divided not interfere with each other suggests that these divisions
to obtain a level that adequately describes the 3-division. are related. A third limitation concerned the maximal sub-
There is a puzzling aspect to the data of Set 3. In both division of the clock unit. It was found that 5-divisions
conditions, the reproduced ratios were greater than 3; in were not reproduced accurately, which suggests that the
relationship between levels is limited to divisions of 2,
Table 3 3, and 4. The first two limitations apply to the domain
Ratios of the Intervals Within the Third Clock Unit of the Stimuli of patterns that can be represented adequately; the third.
of Experiment 2 and Their Mean Reproduction in Two Conditions finding, inaccurate 5-divisions, indicates that the set of
Reproduction clock units from which the most likely selected clock unit
Set Stimuli Mean SD is calculated can be limited to clock intervals with a sub-
1. Different 2:1 2.55 .27 division of less than 5.
Same 2:1 2.09 .20 The present study supports a model of the internal
2. Different 2:1 2.39 .15 representation of temporal patterns in which the perceived
Same 2:1 2.08 .12 clock refers to a higher order time unit. This time unit
3. Different 3:1 3.11 .20 is subdivided into two or three equal parts, which in tum
Same 3:1 3.07 .20 are again subdivided into two or three parts, thus describ-
4. Different 3:1 3.02 .15 ing the intervals in the pattern.
Same 3:1 2.97 .15 A final remark concerns the conclusion that the con-
5. Different 4:1 3.55 .35 ceptual structures necessary to describe the presented com-
Same 4:1 3.51 .30 plex structures were not readily available or elicited. The
 HIERARCHICAL ORGANIZATION OF TEMPORAL PATTERNS
research reported here restricts itself to conceptual struc-
tures used by subjects from a western culture. In other
cultures, different conceptual structures might be used.
Data across cultures could provide insights into the univer-
sal nature of our results.
REFERENCES
EssENS, P. J., & POVEL, D. J. (1985). Metrical and nonmetrical represen-
tations of temporal patterns. Perception & Psychophysics, 37, 1-17.
JONES, M. R. (1976). Time, our lost dimension: Toward a new theory
of perception, attention, and memory. Psychological Review, 83,
323-355.
JONES, M. R. (1981a). Only time can tell: An essay on the topology
of mental space and time. Critical Inquiry, 7, 557-576.
JONES, M. R. (1981b). A tutorial on some issues and methods in serial
pattern research. Perception & Psychophysics, 30, 492-504.
LoNGUET-HIGGINS, H. C. (1978). The perception of music. Interdis-
ciplinary Science Reviews, 3, 148-156.
73
LONGUET-HIGGINS, H. C., & LEE, C. S. (1982). The perception ofmu-
sical rhythms. Perception, 11, 115-128.
LoNGUET-HIGGINS, H. C., & LEE, C. S. (1984). The rhythmic interpre-
tation of monophonic music. Music Perception, 1, 424-441.
MARTIN, J. G. (1972). Rhythmic (hierarchical) versus serial structure
in speech and other behavior. Psychological Review, 79, 487-509.
MICHON, J. A. (1974). Programs and "programs" for sequential pat-
terns in motor behavior. Brain Research, 71, 413-424.
POVEL, D. J. (1981). Internal representation of simple temporal pat-
terns. Journal ofExperimental Psychology: Human Perception & Per-
formance, 7, 3-18.
POVEL, D. J. (1984). A theoretical framework for rhythm perception.
Psychological Research, 45, 315-337.
POVEL, D. J., & ESSENS, P. J. (1985). Perception of temporal patterns.
Music Perception, 2, 411-440.
(Manuscript received May 20, 1985;
revision accepted for publication June 1, 1986.)