The Behavior Analyst 2012, 35, 179–195 No.

2 (Fall)

Understanding Preference Shifts: A Review and

Alternate Explanation of Within-Trial Contrast and
State-Dependent Valuation
James N. Meindl
The University of Memphis
Stimuli that precede aversive events are typically less preferred than stimuli that precede
nonaversive events. It has recently been demonstrated that stimuli that follow less preferred
events may become favored more than stimuli that follow more preferred events. This
phenomenon has been investigated under a variety of names, most commonly, within-trial
contrast and state-dependent valuation. Although this effect has been replicated, there have been
several failures to replicate and it is still little understood. This paper reviews and summarizes
the literature on within-trial contrast and state-dependent valuation. Procedural variations
across studies are identified and discussed. The two current models that explain the phenomenon
are then outlined and the limitations of each model are described. A third explanation is offered
that incorporates the concept of motivating operations. Last, the predictions of all three models
are compared.
Key words: conditioning preference, within-trial contrast, contrast, motivating operations,
state-dependent valuation

Stimuli that are followed by aver-
sive events may become less preferred
than stimuli that are followed by
nonaversive events. It is also possible,
however, for stimulus preference to
be altered by the events that precede
the presentation of the stimulus. This
phenomenon has been investigated
under a variety of names, most com-
monly, within-trial contrast (WTC)
and state-dependent valuation (SDV).
Both WTC and SDV are conceptual
models that essentially describe an
effect wherein exposure to a less
preferred event increases preference
for stimuli that follow that event.
Although each model invokes differ-
ent mechanisms to explain the phe-
nomenon and thus makes different
predictions (discussed later), in gen-
eral both predict that organisms will
demonstrate a preference for stimuli
that follow less preferred events rela-
tive to stimuli that follow more
preferred events.
In one of the first studies examining
this phenomenon, Clement, Feltus,
Kaiser, and Zentall (2000) exposed
eight pigeons to two chain-schedule
conditions that differed by the
amount of effort required.1 In the
first condition (see Figure 1), pigeons
were exposed to a fixed-ratio (FR) 1
schedule with a lit center key (initial

I thank Jonathan Ivy, Nancy Neef, and

Manish Vaidya for their thoughtful comments
and suggestions regarding this paper. The
contents of this article were developed under a
grant from the U.S. Department of Educa-
tion, OSEP (H325DO60032) (N. A. Neef,
Principal Investigator). These contents do not
necessarily represent the policy of the U.S.
Department of Education, OSEP, and no
endorsement by the federal government
should be assumed.
Correspondence concerning this article
should be addressed to the author at the
University of Memphis, 400A Ball Hall,
Memphis, Tennessee 38152 (e-mail: jnmeindl@
memphis.edu).
1
Throughout this paper, chain schedules
will be described as consisting of three
components (an initial component, middle
component, and terminal component). The
term initial component will refer to the begin-
ning of the chain, middle component will refer
to the event manipulated by the researchers
(e.g., response requirement, effort, delay), and
terminal component will refer to the conditions
that follow the manipulated event. Although
the usage is unconventional (cf. Clement et al.,
2000), explicit identification of a middle
component will allow discussion of a larger
variety of events than is possible using
conventional chain schedule descriptions.

179
180 JAMES N. MEINDL
Figure 1. Training trials adapted from Clement et al. (2000). A key peck to Circ produced
either an FR 1 or FR 20 schedule followed by discrimination trials. A key peck to S+ resulted in
food reinforcement. A key peck to S2 resulted in no food reinforcement.
component). Completion of this com-
ponent led to another FR 1 schedule
(middle component) followed by a
two-key discrimination task (terminal
component). In the terminal compo-
nent, a peck to the red key (S+FR1)
resulted in food reinforcement where-
as a peck to the yellow key (S2FR1)
resulted in no food reinforcement.
The second condition consisted of an
identical initial component, followed
by an FR 20 in the middle compo-
nent, followed by a similar two-key
discrimination task in which a peck to
the green key (S+FR20) resulted in
food reinforcement whereas a peck to
the blue key (S2FR20) resulted in no
food reinforcement. After repeated
exposure to both conditions, prefer-
ence for the various terminal compo-
nent stimuli was assessed via a paired-
stimulus preference assessment that
compared S+FR1 against S+FR20 and
S2FR1 against S2FR20. Results indi-
cated a preference for the S+FR20 and
the S2FR20 (both stimuli that had
previously followed the high-effort
condition).
Although there have been multiple
replications, the finding is not entire-
ly reliable, and there have been
numerous failures to replicate. Fur-
ther, there are still lingering questions
as to the cause of the change in
preference and whether any middle-
component events can produce the
effect. As a result, two competing
models have been offered to explain
the phenomenon: WTC and SDV.
The purpose of this paper is three-
fold. First, the literature on WTC
and SDV will be reviewed and the
results will be discussed in terms of
subject type and initial, middle, and
terminal components. Differences in
procedures will be delineated, as will
inconsistencies in findings. Second,
the current models that explain the
phenomenon will be contrasted, and
their limitations will be described.
Third, an alternative explanation of
the findings will be presented, and the
predictions of all three models will be
evaluated.
METHOD
Two literature searches were con-
ducted using the databases Psyc-
INFO and PubMed. The key words
used were within-trial contrast for the
first search and state-dependent valu-
ation for the second. All resulting
peer-reviewed articles were retained.
Of these articles, a search of the
references was conducted and studies
that investigated WTC or SDV or
used similar procedures were includ-
ed in the list of articles. Excluded
were articles that focused only on
cognitive dissonance or justification
of effort, because these studies typi-
cally employed different procedures
 UNDERSTANDING PREFERENCE SHIFTS
or did not directly measure prefer-
ence but rather relied on preference
rating scales. Also excluded was
research on the concorde fallacy and
sunk cost effects (see Arkes & Ayton,
1999, for discussion). Although these
concepts are somewhat related to WTC
and SDV, the procedures are quite
dissimilar and may not examine the
same behavior-change phenomenon.
RESULTS AND DISCUSSION
Generality of the Findings
In total, 23 articles were identified
containing a total of 38 separate
experiments. Of these 38 experiments,
22 (across 18 separate articles) dem-
onstrated an increase in preference
for a stimulus that followed a less
rather than more preferred event, and
16 (across 8 separate articles) failed
to replicate the phenomenon. Note
that some articles contained both
replications and failures to replicate.
Of the 23 total articles, six appeared
to interpret the phenomenon from
the conceptual perspective of SDV
(Aw, Holbrook, Burt de Perera, &
Kacelnik, 2009; Kacelnik & Marsh,
2002; Marsh, Schuck-Paim, & Kacel-
nik, 2004; Pompilio & Kacelnik,
2005; Pompilio, Kacelnik, & Behmer,
2006; Waite & Passino, 2006), and 15
interpreted the phenomenon from
the perspective of WTC (Alessandri,
Darcheville, Delevoye-Turrell, &
Zentall, 2008; Alessandri, Darche-
ville, & Zentall, 2008; Arantes &
Grace, 2008; Clement et al., 2000;
Clement & Zentall, 2002; DiGian,
Freidrich, & Zentall, 2004; Friedrich,
Clement, & Zentall, 2005; Friedrich
& Zentall, 2004; Gipson, Miller,
Alessandri, & Zentall, 2009; Klein,
Bhatt, & Zentall, 2005; O’Daly,
Meyer, & Fantino, 2005; Singer,
Berry, & Zentall, 2007; Vasconcelos
& Urcuioli, 2008, 2009; Vasconcelos,
Urcuioli, & Lionello-DeNolf, 2007).
One study directly compared empir-
ical predictions from both conceptual
frameworks (Aw, Vasconcelos, &
Kacelnik, 2011). Finally, a clear
181
conceptual perspective was not iden-
tifiable for one study (Armus, 2001).
Because the procedures used in these
studies were quite similar despite
different conceptual underpinnings,
the articles will be collapsed and
discussed together. (See Appendix
A for a breakdown of each article
and to examine effect by individual
experiment.)
Subject types. Of the 22 experi-
ments that successfully demonstrated
the effect, 18 were conducted with
various nonhumans including pi-
geons, locusts, grasshoppers, rats,
banded tetras, and starlings. The
remaining four experiments demon-
strated the effect with humans, both
children and adults. Of the 16 exper-
iments that were unable to replicate
the findings, all were conducted with
nonhuman organisms including star-
lings, pigeons, or rats. These data
suggest that the phenomenon is
general and not a species-specific
characteristic.
Middle components. Across the 22
experiments that successfully demon-
strated the effect, a variety of events
have been programmed for or re-
quired in the middle component. For
example, the effect has been docu-
mented when the preceding events
were high versus low effort (Alessan-
dri, Darcheville, Delevoye-Turrell, &
Zentall, 2008; Aw et al., 2011; Cle-
ment et al., 2000; Clement & Zentall,
2002; Friedrich & Zentall, 2004;
Kacelnik & Marsh, 2002; Klein et
al., 2005), high versus low probabili-
ties of reinforcement (Clement &
Zentall, 2002; Gipson et al., 2009),
short versus long delays to the termi-
nal component Alessandri, Darche-
ville, Delevoye-Turrell, et al., 2008;
Alessandri, Darcheville, & Zentall,
2008; Clement et al., 2000; DiGian
et al., 2004; O’Daly et al., 2005),
preferred versus less preferred sched-
ules of reinforcement (Singer et al.,
2007), the absence or presence of
reinforcement (Friedrich et al.,
2005), and low versus high states of
food deprivation (Aw et al., 2009;
182 JAMES N. MEINDL
Marsh et al., 2004; Pompilio &
Kacelnik, 2005; Pompilio et al.,
2006; Vasconcelos & Urcuioli, 2008).
Although the effect has been most
robustly documented with food dep-
rivation, in each of the above cases,
preference was increased for stimuli
that followed the less preferred event.
Thus, the effect does not appear to be
limited to conditions that involve
differential response requirements
(or more generally, effort) alone.
Although the effect has been pro-
duced with a variety of events in
middle components, there have also
been many failures to replicate the
effect. Studies that have failed to
produce the effect have examined
middle components such as high
versus low effort (Arantes & Grace,
2008; Armus, 2001; Friedrich &
Zentall, 2004; Vasconcelos & Ur-
cuioli, 2009; Vasconcelos et al.,
2007; Waite & Passino, 2006), long
versus short delay (Aw et al., 2011),
and high versus low states of food
deprivation (Vasconcelos & Urcuioli,
2008).
Explanations for replication failures.
Explanations for these failures to
replicate the phenomenon have been
numerous. Singer et al. (2007) sug-
gested that the effect is slow to
develop. Zentall (2008) suggested a
variety of possible reasons for failure
to replicate, including that overtrain-
ing is required, that the terminal
component must be contiguous with
the middle component, and that prior
exposure to lean schedules influences
the effect. Finally, Aw et al. (2011)
suggested that the effect occurs only
when the middle component requires
energy expenditure.
For each suggested explanation,
however, there is at least one study
that demonstrates that explanation to
be insufficient. First, Arantes and
Grace (2008) provided an amount of
training equal to that used in other
successful studies and were unable to
reproduce the results. Further, Vas-
concelos and Urcuioli (2009) provided
extensive overtraining, but although
they noted a tendency towards pref-
erence change, this tendency was not
statistically significant. Second, an
effect has been found when using a
delay as the less preferred event (e.g.,
Alessandri, Darcheville, & Zentall,
2008), which would seem to counter
both the argument concerning conti-
guity between middle and terminal
components and the argument that
energy expenditure is required. Final-
ly, researchers have used experimen-
tally naive pigeons and employed
overtraining but still failed to replicate
previous findings (e.g., Vasconcelos &
Urcuioli, 2009).
There are other possible explana-
tions for some replication failures.
Armus (2001), for example, conduct-
ed experiments using two differently
flavored pellets (grape and bacon) as
terminal components following dif-
ferent amounts of effort. Repeated
exposure to different food items may
increase preference (Wardle, Herrera,
Cooke, & Gibson, 2003), and it is
possible that this repeated exposure
masked or influenced the results of
the experiment.
Procedural Variations
Across Experiments
There are a variety of procedural
differences across the various exper-
iments that have examined the phe-
nomenon. It is possible that these
variations contribute to the differenc-
es in experimental findings. The
following sections describe several
procedural variations regarding the
presentation of the initial and termi-
nal components across the reviewed
studies.
Initial component. A stimulus that
precedes a less preferred event may
become a conditioned aversive stim-
ulus and function as a punisher (e.g.,
Vorndran & Lerman, 2006). Because
a change in preference for stimuli
presented in the terminal component
is presumed to be directly related to
preference for the event in the middle
component, it is necessary to be able to
 UNDERSTANDING PREFERENCE SHIFTS
attribute the preference change to those
two components alone. If the initial
component is a less preferred or
aversive event, this may confound
interpretations by either adding or
subtracting from the effect. It is there-
fore important to distinguish between
studies that used distinct initial com-
ponents (in which differential prefer-
ence could develop) and those that used
identical initial components.
Of the experiments that successful-
ly demonstrated the phenomenon,
10 used distinct initial components
(Alessandri, Darcheville, Delevoye-
Turrell, et al., 2008, Phases 1 and 2;
Alessandri, Darcheville, & Zentall,
2008; Aw et al., 2011, Experiment 3;
Clement & Zentall, 2002, Experi-
ments 1, 2, and 3; Friedrich et al.,
2005; O’Daly et al., 2005, Experiment
2; Singer et al., 2007), 4 used identical
initial components (Clement et al.,
2000; Friedrich & Zentall, 2004,
Experiment 1; Gipson et al., 2009;
Klein et al., 2005), 2 used both
distinct and identical initial compo-
nents for different groups (DiGian
et al., 2004; O’Daly et al., 2005,
Experiment 1), and 6 did not include
initial components (Aw et al., 2009;
Kacelnik & Marsh, 2002; Marsh et
al., 2004; Pompilio & Kacelnik, 2005;
Pompilio et al., 2006; Vasconcelos &
Urcuioli, 2008, Experiment 1). Those
that did not include initial compo-
nents investigated the effects of dif-
ferent levels of food deprivation, and
an initial component was not feasible.
Of the 16 experiments that were
unable to replicate the phenomenon,
3 used distinct initial components
(Aw et al., 2011, Experiment 2;
Vasconcelos & Urcuioli, 2009, Ex-
periments 1 and 2), 9 used identical
initial components (Arantes & Grace,
2008, Experiments 1 and 2; Aw et al.,
2011, Experiment 1; Friedrich &
Zentall, 2004, Experiment 2; Vascon-
celos et al., 2007, Experiments 1
through 5), 1 used both distinct and
identical initial components (Vascon-
celos et al., 2007, Experiment 6), and
183
3 did not include initial components
(Armus, 2011; Vasconcelos & Ur-
cuioli, 2008, Experiment 2; Waite &
Passino, 2006).
Of the 13 experiments that used
identical stimuli in the initial compo-
nents, four produced an effect and
nine did not. Of the 13 experiments
that used different stimuli in the
initial components, 10 produced an
effect and three did not. Thus,
whether the stimuli used in the initial
component influence the contrast
effect is currently unclear, but the
data suggest the possibility that they
do. Studies directly examining this
possibility include, for example, Di-
Gian et al. (2004) who used different
initial components (vertical or hori-
zontal lines) for one group of pigeons
and identical initial components
(white keys) for a different group of
pigeons. Although both groups sig-
nificantly preferred the stimulus that
followed longer delays, the first
group (different initial components)
displayed a greater degree of prefer-
ence change than the second group
(identical initial components). This
may indicate that the initial compo-
nent exerts some effect in the condi-
tioning process, although such a
conclusion is speculative at this point.
Terminal component. The presenta-
tion of terminal component stimuli
varied among studies primarily based
on whether the researchers presented
a single stimulus after each condition
or presented two stimuli together in a
discrimination task. In 12 of the 22
experiments that demonstrated an
effect, a discrimination task followed
the middle component (Alessandri,
Darcheville, Delevoye-Turrell, et al.,
2008, Phases 1 and 2; Alessandri,
Darcheville, & Zentall, 2008; Cle-
ment et al., 2000; Clement & Zentall,
2002, Experiments 1 through 3;
DiGian et al., 2004; Friedrich et al.,
2005; Gipson et al., 2009; Klein et al.,
2005; Singer et al., 2007, Experi-
ment 2). The remaining 10 studies
presented a single stimulus after each
condition. Of the 16 experiments that
184 JAMES N. MEINDL
failed to replicate the effect, 11
employed a discrimination task fol-
lowing the middle component (Aran-
tes & Grace, 2008, Experiments 1
and 2; Vasconcelos & Urcuioli, 2008,
Experiment 2; Vasconcelos & Ur-
cuioli, 2009, Experiments 1 and 2;
Vasconcelos et al., 2007, Experiments
1 through 6). The remaining five
studies presented a single stimulus
after each condition. In summary, of
the 23 studies that presented a
discrimination task in the terminal
component, 12 successfully produced
the effect. Of the 15 studies that
presented a single stimulus in the
terminal component, 10 produced
the effect. Thus it appears as though
the phenomenon is more frequently
seen when terminal components in-
volve single stimulus presentation
rather than discrimination tasks.
Experiments that employed a dis-
crimination task in the terminal
component have produced conflict-
ing results. Of the studies that repli-
cated the phenomenon, some have
shown that both the S+ and S2
following the less preferred event
(S+LP; S2LP) are equally likely to
become preferred relative to the S+
and S2 following the more preferred
event (Clement & Zentall, 2002;
Gipson et al., 2009; Singer et al.,
2007), and Clement et al. (2000)
found an even greater preference for
S2LP. These findings suggest that
relative preference for the preceding
event is responsible for the effect
rather than the events that follow
the terminal component (i.e., rein-
forcement or no reinforcement).
Other research, however, has dem-
onstrated that the effect is weaker
with the S2LP than the S+LP (Fried-
rich et al., 2005) or have found no
effect with the S2LP (Alessandri,
Darcheville, & Zentall, 2008; Clement
& Zentall, 2002; DiGian et al., 2004;
Klein et al., 2005). These results are
difficult to interpret and may suggest
that the reinforcement that follows
the S+ somehow influences the effect.
An alternative explanation, howev-
er, might be that in order for prefer-
ence to change, the organism must
respond to the stimulus that follows
the middle component. Arantes and
Grace (2008) and Vasconcelos et al.
(2007) propose that because the S+
and S2 were presented in a discrim-
ination task, the organism learned to
consistently select the S+ stimulus and
avoid the S2. They suggest that the
presentation of stimuli in a discrimi-
nation task may produce an effect
with the S+ but may inhibit the effect
with the other stimulus. Currently, it
is unclear whether this is the case or
whether the addition of reinforcement
after a terminal component somehow
influences the effect.
More straightforward examples of
the effect are evident in the 10
successful experiments that did not
impose a discrimination task in the
terminal component (Aw et al., 2009;
Aw et al., 2011, Experiment 3; Fried-
rich & Zentall, 2004, Experiment 1;
Kacelnik & Marsh, 2002; Marsh et
al., 2004; O’Daly et al., 2005, Exper-
iments 1 and 2; Pompilio & Kacelnik,
2005; Pompilio et al., 2006; Vascon-
celos & Urcuioli, 2008, Experiment
1). In these studies the terminal
component was a single stimulus
(e.g., differently colored key, specific
arm of a Y maze, etc.). After training,
preference was measured by present-
ing both terminal components simul-
taneously and recording the organ-
ism’s choice between components. In
each of these studies, a significant
preference was found for the stimulus
that had previously followed less
preferred events, with the exception
of Aw et al. (2011), who found an
effect only when the middle compo-
nent involved effort rather than delay.
Procedural Problems with
Reviewed Studies
Because changes in preference for
terminal component stimuli are pur-
portedly related to differential pref-
erence for the middle component
 UNDERSTANDING PREFERENCE SHIFTS
stimuli, it seems important that base-
line preference for both components
be established prior to the onset of an
experiment. Surprisingly, however, of
the 38 experiments reviewed (includ-
ing both replications and failures
to replicate) only three (Alessandri,
Darcheville, Delevoye-Turrell, et al.,
2008, Experiments 1 and 2; Singer
et al., 2007) measured preference for
the middle components. Friedrich and
Zentall (2004) did measure middle-
component preference, but only after
training was complete.
It may be tempting to assume that
more effort or longer delays will
always be less preferred; however,
this may not always be true. For
example, Alessandri, Darcheville,
Delevoye-Turrell, et al. (2008) em-
ployed middle components that re-
quired participants to press a button
with varying amounts of force and
for varying lengths of time. For two
participants, the less preferred condi-
tion was not the least effort/shortest
delay condition. Without prior mea-
surement, results for these two par-
ticipants might have been erroneous-
ly interpreted.
Another concern is that only two
of the 38 experiments assessed pref-
erence for terminal component stim-
uli prior to training (Friedrich &
Zentall, 2004, Experiments 1 and 2).
Without knowing initial preference, it
is difficult to determine the impact of
training. If preference for a stimulus
was initially low and then shifted to
high, this would indicate a very
strong effect. If, on the other hand,
preference for the stimulus was ini-
tially somewhat high and then shifted
higher, this would indicate a fairly
weak effect. Without measuring pref-
erence for middle and terminal com-
ponents prior to training, it is diffi-
cult to draw strong conclusions
regarding the phenomenon. If we
are to suggest that preference for
terminal components is influenced by
preference for middle components, it
is of utmost importance that we
185
establish these preferences before
exposure to training.
DIFFERENT MODELS TO
EXPLAIN THE EFFECTS
The Within-Trial Contrast Model
The WTC model (see Figure 2)
was first proposed by Zentall (2005)
and generally assumes that when an
organism is exposed to a less pre-
ferred event there is a negative
change in the organism’s hedonic
state (H 2 DH). This negative change
is directly proportional to the degree
to which the event is less preferred.
When a reinforcing stimulus is pre-
sented following this negative shift,
there is a positive shift in the hedonic
state of the organism. The contrast,
then, is between the organism’s he-
donic state before and after the
presentation of the reinforcing stim-
ulus. The degree to which preference
is changed depends on the amount of
positive shift, and therefore also on
the degree to which one event is
preferred more or less than another
(cf. relative values in Figure 2).
Limitations of the within-trial
contrast model. There are several
problems with the WTC model. Aw
et al. (2011) outline many similar
limitations; however, they deserve
mention here in order to contrast
the various models. The first limita-
tion is that the model assumes a
hedonic state and uses changes in this
state (both absolute and relative) as
the primary events responsible for
contrast. Hedonic state, however, is a
term that only vaguely refers to an
organism’s well-being, and there is
currently no agreed-upon definition
(Pompilio & Kacelnik, 2005). Hedon-
ic state seems to function much the
same as a hypothetical construct, and
there is no method of objectively
determining an organism’s hedonic
state, much less detecting changes in
that state. Although the term hedonic
state, and assumptions of changes in
this state, may be convenient, it is
186 JAMES N. MEINDL

Figure 2. ‘‘A model based on change in relative hedonic value, proposed to account for
within-trial contrast effects. According to the model, trials begin with a relative hedonic state, H;
key pecking results in a negative change in hedonic state, H 2 DH1 for FR 1 and H 2 DH20 for
FR 20; obtaining a reinforcer results in a positive change in hedonic state, H + DHRf; the net
change in hedonic state depends on the difference between H + DHRf and H 2 DH1 on an FR 1
and between H + DHRf and H 2 DH20 on an FR 20 trial.’’ (Zentall, 2005, p. 280).

unclear whether hedonic state refers relative value of a reinforcer is

to any measurable state of being.
A second problem with this model
is that it describes an upward shift in
hedonic state as the result of the
presentation of a reinforcer after a
less preferred event. Recall, however,
that several researchers used a dis-
crimination task as the terminal
component and found an effect with
both the S+ and S2 (Clement et al.,
2000; Clement & Zentall, 2002; Gip-
son et al., 2009; Singer et al., 2007).
Because the S2 was never associated
with a reinforcer, there should be no
increase in hedonic state and hence
no shift in preference for the S2
stimulus.
Due to these limitations, the cur-
rent WTC model appears to be an
inadequate explanation. Although
the WTC model affords some predic-
tion, the model is less than ideal at
providing a scientific explanation of
the phenomenon.
The State-Dependent Valuation Model
Kacelnik and Marsh (2002) pro-
posed the SDV as an alternative
model that hypothesizes that the
directly related to the energetic state
or fitness of the organism at the point
of reinforcer delivery. A reinforcer
delivered at a point of low energy
reserves is presumed to be relatively
more valuable than that same rein-
forcer delivered at a point of higher
energy reserves. The model further
assumes that with repeated exposures
to a food item under states of low
energy reserves, the utility value of
the food item is somehow represented
in memory (Pompilio & Kacelnik,
2005).
Limitations of the state-dependent
valuation model. One limitation of the
SDV model is that it assumes that the
function of the middle components is
to differentially depress energy re-
serves. It is unclear whether this is
always the case, however, because the
effect has been successfully demon-
strated with middle components that
would not be expected to differen-
tially depress energy reserves, such as
delay lengths (DiGian et al., 2004),
different schedules of reinforcement
(Singer et al., 2007), or anticipated or
unanticipated effort (Clement & Zen-
tall, 2002). If energy reserves were not
 UNDERSTANDING PREFERENCE SHIFTS
differentially depressed, however, the
terminal components would not have
differential fitness value, and the
effect would not be predicted.
A second limitation is that several
studies have shown changes in prefer-
ence when the terminal components
were followed by stimuli with presum-
ably no capacity to increase energy
reserves. Alessandri, Darcheville, and
Zentall (2008), for example, provided
children with short song segments or
segments of a cartoon after successful
discrimination in the terminal compo-
nent. Similarly, Klein et al. (2005) had
a computer screen display the words
‘‘correct’’ or ‘‘incorrect’’ during dis-
crimination. Furthermore, as noted in
the limitations of the WTC model, an
increase in preference has been ob-
served with S2 that was not associat-
ed with any reinforcement. Thus, it
appears that for some of the successful
demonstrations, the SDV model pro-
vides an insufficient explanation.
ALTERNATIVE EXPLANATION:
MOTIVATING OPERATIONS
AND THE FUNCTION-
ALTERING EFFECT
As currently conceptualized, moti-
vating operations (MOs) are stimulus
conditions or events that produce a
momentary change in the reinforcing
effectiveness of other stimuli (the
value-altering effect) and a momen-
tary change in the frequency of
behaviors (the behavior-altering ef-
fect) that have functioned to produce
those stimuli (Michael, 2004). Thus,
an MO is not associated with differ-
ential availability of consequences,
but rather produces a temporary
change in stimulus value and the
probability of behavior that produces
that stimulus.
The function-altering effect de-
scribes a conditioning process where-
by, due to a particular learning
history, the function of specific stim-
uli is altered in the presence of other
stimuli (Schlinger & Blakely, 1994).
If, for example, food is provided for
187
key pecking under a state of food
deprivation (an establishing opera-
tion), key pecking is more likely in
the future because the evocative effect
of the establishing operation has been
altered. The function-altering effect
has been to modify the function of
the establishing operation. In this
case, the evocative effect of food
deprivation is to evoke key pecking.
Let us now apply the concept of
MOs and function-altering effects to
the typical presentation of the initial,
middle, and terminal components in
the reviewed studies. In a typical
study on effort, for example, a pigeon
might be exposed to both of the
following conditions in an alternating
fashion:
Condition 1: Initial Component 1 (FR 1) R
Middle Component 1 (FR 5) R Terminal
Component 1 (red key followed by food
reinforcement)
Condition 2: Initial Component 2 (FR 1) R
Middle Component 2 (FR 30) R Terminal
Component 2 (blue key followed by food
reinforcement)
In both conditions, engaging in the
initial component produces differen-
tial effort outcomes: low effort (FR
5) or high effort (FR 30). Assuming
that the pigeon would prefer not to
engage in any effort expenditure,
both Middle Components 1 and 2
may be considered MOs in that they
both function to increase the value of
stimuli that terminate the middle
component. With repeated exposure
to training, the function of these
establishing operations is altered.
However, due to differences in the
magnitude of the required effort in
both events, Middle Component 2
(FR 30) is less preferred than Middle
Component 1 (FR 5). Therefore,
although the stimuli that follow both
Middle Components 1 and 2 should
function as conditioned reinforcers,
the stimuli associated with the termi-
nation of Middle Component 2
should be valued more than the
stimuli associated with the termina-
tion of Middle Component 1. The
188 JAMES N. MEINDL
entire procedure, therefore, has
served to expose the pigeon to a less
preferred stimulus condition (the
middle component) and to associate
the termination of that condition
with the presentation of another
stimulus (the terminal component).
It has previously been noted that
the terminal component of a chain
schedule may become a conditioned
reinforcer because it is consistently
contiguous with the delivery of a
primary reinforcer at the end of the
chain (Fantino, 1977). The explana-
tion described here asserts that the
same effect occurs in these chain
schedules except that the terminal
component becomes a conditioned
reinforcer not through contiguity
with the positive reinforcer but
through contiguity with the termina-
tion of the middle component. In
essence, a stimulus that terminates a
less preferred event is a reinforcer,
and the less preferred the event, the
more valuable the stimulus.
The MO/function-altering expla-
nation is more parsimonious than
the SDV model because it does not
assume that some events (e.g.,
6-s delays; differential reinforcement
of other behavior vs. fixed-interval
schedules) function to decrease an
organism’s energy reserves. Nor does
it assume that the presentation of
cartoons or affirmations functions to
increase an organism’s biological
fitness. The MO interpretation is also
more parsimonious than the WTC
model because it is able to specify the
precise mechanism that is responsible
for altering preference (i.e., MOs) as
opposed to relying on unmeasured
changes in a hypothetical and ill-
defined construct (i.e., hedonic state).
In addition, the MO interpretation
could help to explain some of the
inconsistent findings regarding the
effect. The most reliable demonstra-
tion of the effect has been when levels
of food deprivation were manipulat-
ed, whereas the least reliable demon-
strations were those that used events
such as delay. The MO explanation
would predict that changes in prefer-
ence for the terminal components are
directly related to the degree to which
preference for the middle compo-
nents differs. If one middle compo-
nent was highly aversive and the
other only mildly nonpreferred, a
large preference change would be
expected, and this preference change
would diminish as the two middle
components became equally aversive.
It is possible that there is simply a
larger discrepancy in preferences for
levels of food deprivation compared
to different lengths of delay.
Further, this explanation is able to
account for findings regarding in-
creased preference for the S2LP. As
noted earlier, the WTC and SDV
models predict an increase in prefer-
ence when a reinforcer is delivered
that differentially increases hedonic
states or energetic reserves. Because
the S2 was never associated with
reinforcement, neither model would
predict preference change. On the
other hand, the MO explanation
would predict preference change be-
cause the S2LP was associated with
the termination of a relatively less
preferred event.
DIFFERENT PREDICTIONS
YIELDED BY EACH MODEL
If an organism was exposed to two
training conditions with different
middle components (long delay and
short delay, both followed by some
stimulus), the different models would
yield different predictions. The WTC
model would predict preference for
the stimulus that followed the long
delay relative to the stimulus that
followed the short delay. The SDV
model would not predict differentiat-
ed preference because neither delay
condition resulted in energetic expen-
diture. According to the MO inter-
pretation, a prediction could only be
made if the two delay conditions were
demonstrated to be differentially
preferred. That is, if the long delay
was less preferred than the short
 UNDERSTANDING PREFERENCE SHIFTS
delay, preference should shift to-
wards the stimulus following the long
delay. If the short delay was less
preferred than the long delay, the
opposite prediction would be made.
Furthermore, the MO interpreta-
tion could make novel predictions to
which the WTC and SDV models
could not as easily speak. For exam-
ple, if instead of identical middle
components differing by one dimen-
sion (e.g., 5-s delay vs. 20-s delay),
the middle components were a less
preferred condition involving energy
expenditure (e.g., running) versus a
less preferred condition involving
energy gain (e.g., eating an unpleas-
ant-tasting food item) the predictions
of the WTC and SDV are unclear.
The WTC model would not neces-
sarily be able to quantify which
condition resulted in a greater de-
crease in hedonic state and could
only make this determination after
the fact. The SDV model is predicat-
ed on the expectance of energy
expenditure, so the unpleasant-tast-
ing food item would not fit with the
model. On the contrary, the MO
explanation would merely require
that one condition be shown to be
less preferred relative to the other to
make a prediction as to preference
change.
Finally, the MO explanation may
be extended to situations in which
one middle component is nonpre-
ferred and the other is preferred. If,
for example, one condition involved
an unpleasant tone and the other
involved watching a preferred movie
segment, the MO explanation would
suggest increased preference for the
stimulus following the unpleasant
tone and decreased preference for
the stimulus following the preferred
movie clip (being associated with the
termination of a preferred event). It is
unclear what outcome would be
predicted by the WTC model, and
the SDV model would predict no
preference change because no energy
was expended in either condition.
189
CONCLUSION
The precise circumstances neces-
sary to produce the preference-
change phenomenon described in this
paper are not fully understood at this
time. The effect has been replicated
multiple times across a variety of
species and middle components, yet
there have also been numerous fail-
ures to replicate. Some of the dis-
crepancies in findings may be due to
procedural differences across studies
(e.g., the presentation of a discrimi-
nation task vs. single stimulus pre-
sentation; identical vs. distinct initial
components). It is also possible that
these conflicts are due to procedural
problems seen across most of the
studies reviewed (e.g., a failure to
measure preference for middle and
terminal components prior to train-
ing). Further complicating our un-
derstanding of the phenomenon is
that the two models currently offered
to explain the findings both make
different predictions and are unable
to account for all of the research
findings. This paper offers an alter-
native explanation to the WTC and
SDV models. This alternative expla-
nation relies on the concept of MOs
and function-altering effects and sug-
gests that the terminal components
are conditioned as reinforcers
through contiguity with the termina-
tion of the middle components. Al-
though this account remains specula-
tive at this point, it is conceptually
systematic, is able to account for
inconsistent research findings, and is
able to make novel predictions.
The studies reviewed in this paper,
along with the alternative explana-
tion, have important implications for
behavior analysis. One implication
concerns the design of behavioral
interventions. When these programs
are created, reinforcers are selected
and then typically programmed to be
delivered on some schedule. Al-
though much attention is given to
the effect a reinforcer has on an
organism’s engagement with a sched-
190 JAMES N. MEINDL
ule, very little attention is given to the
effect the schedule, and correspond-
ing work requirements, have on the
value of the reinforcer. The studies
reviewed here indicate that the work
requirement may alter the value of
the corresponding reinforcer. Care,
therefore, should be exerted when
selecting and delivering reinforcers
because the schedule requirements
may either increase or decrease the
value of those stimuli.
A second implication is related to
the current understanding of the
interaction between MOs and the
function-altering effect. The value-
altering effect of an establishing
operation is understood as ‘‘an in-
crease in the current [italics added]
effectiveness of some stimulus, object,
or event as reinforcement’’ (Cooper,
Heron, & Heward, 2007, p. 376).
Food deprivation is presumed to
momentarily increase the effectiveness
of food as a reinforcer. When the
level of food deprivation subsides, the
effectiveness of food as a reinforcer
diminishes. The research reviewed
here, however, suggests that the
function-altering effect interacts with
MOs. That is, the food is made more
valuable when it is presented during
food deprivation, and that increase
in value persists into the future.
The finding that less preferred
events alter preference for subsequent
stimuli warrants further inquiry. Fu-
ture research into this phenomenon
may help explain how stimulus pref-
erences develop with greater preci-
sion than is currently available, and
may allow a greater understanding of
how the environment and behavior
interact and affect one another.
REFERENCES
Alessandri, J., Darcheville, J.-C., Delevoye-
Turrell, Y., & Zentall, T. R. (2008).
Preference for rewards that follow greater
effort and greater delay. Learning & Behav-
ior, 36, 352–358.
Alessandri, J., Darcheville, J.-C., & Zentall,
T. R. (2008). Cognitive dissonance in
children: Justification of effort or contrast?
Psychonomic Bulletin & Review, 15, 673–677.
Arantes, J., & Grace, R. C. (2008). Failure to
obtain value enhancement by within-trial
contrast in simultaneous and successive
discriminations. Learning & Behavior, 36,
1–11.
Arkes, H. R., & Ayton, P. (1999). The sunk
cost and concorde effects: Are humans less
rational than lower animals. Psychological
Bulletin, 125, 591–600.
Armus, H. L. (2001). Effect of response effort
on the reward value of distinctively flavored
food pellets. Psychological Reports, 88,
1031–1034.
Aw, J. M., Holbrook, R. I., Burt de Perera, T.,
& Kacelnik, A. (2009). State-dependent
valuation learning in fish: Banded tetras
prefer stimuli associated with greater past
deprivation. Behavioral Processes, 81, 333–
336.
Aw, J. M., Vasconcelos, M., & Kacelnik, A.
(2011). How costs affect preferences: Exper-
iments on state dependence, hedonic state
and within-trial contrast in starlings. Animal
Behaviour, 81, 1117–1128.
Clement, T. S., Feltus, J. R., Kaiser, D. H., &
Zentall, T. R. (2000). ‘‘Work ethic’’ in
pigeons: Reward value is directly related to
the effort or time required to obtain the
reward. Psychonomic Bulletin & Review, 7,
100–106.
Clement, T. S., & Zentall, T. R. (2002).
Second-order contrast based on the expec-
tation of effort and reinforcement. Journal
of Experimental Psychology, 28, 64–74.
Cooper, J. O., Heron, T. E., & Heward, W. L.
(2007). Applied behavior analysis (2nd ed.).
Upper Saddle River, NJ: Pearson Education.
DiGian, K. A., Freidrich, A. M., & Zentall,
T. R. (2004). Discriminative stimuli that
follow a delay have added value for pigeons.
Psychonomic Bulletin & Review, 11,
889–895.
Fantino, E. (1977). Conditioned reinforce-
ment: Choice and information. In W. K.
Honig & J. E. R. Staddon (Eds.), Handbook
of operant behavior (pp. 313–363), Engle-
wood Cliffs, NJ: Prentice Hall.
Friedrich, A. M., Clement, T. S., & Zentall,
T. R. (2005). Discriminative stimuli that
follow the absence of reinforcement are
preferred by pigeons over those that follow
reinforcement. Learning & Behavior, 33,
337–342.
Friedrich, A. M., & Zentall, T. R. (2004).
Pigeons shift their preference toward loca-
tions of food that take more effort to
obtain. Behavioral Processes, 67, 405–415.
Gipson, C. D., Miller, H. C., Alessandri, J. J. D.,
& Zentall, T. R. (2009). Within-trial contrast:
The effect of probability of reinforcement in
training. Behavioral Processes, 82, 126–132.
Kacelnik, A., & Marsh, B. (2002). Cost can
increase preference in starlings. Animal
Behavior, 63, 245–250.
Klein, E. D., Bhatt, R. S., & Zentall, T. R.
(2005). Contrast and justification of effort.
 UNDERSTANDING PREFERENCE SHIFTS
Psychonomic Bulletin & Review, 12, 335–
339.
Marsh, B., Schuck-Paim, C., & Kacelnik, A.
(2004). Energetic state during learning
affects foraging choices in starlings. Behav-
ioral Ecology, 15, 396–399.
Michael, J. (2004). Concepts and principles of
behavior analysis. Kalamazoo, MI: Associ-
ation for Behavior Analysis.
O’Daly, M., Meyer, S., & Fantino, E. (2005).
Value of conditioned reinforcers as a
function of temporal context. Learning and
Motivation, 36, 42–59.
Pompilio, L., & Kacelnik, A. (2005). State-
dependent learning and suboptimal choice:
When starlings prefer long over short delays
to food. Animal Behavior, 70, 571–578.
Pompilio, L., Kacelnik, A., & Behmer, S. T.
(2006). State-dependent learned valuation
drives choice in an invertebrate. Science,
311, 1613–1615.
Schlinger, H. D., & Blakely, E. (1994). A
descriptive taxonomy of environmental
operations and its imlpications for behav-
ior analysis. The Behavior Analyst, 17,
43–57.
Singer, R. A., Berry, L. M., & Zentall, T. R.
(2007). Preference for a stimulus that
follows a relatively aversive event: Con-
trast or delay reduction. Journal of the
Experimental Analysis of Behavior, 87,
275–285.
Vasconcelos, M., & Urcuioli, P. J. (2008).
Deprivation level and choice in pigeons: A
191
test of within-trial contrast. Learning &
Behavior, 36, 12–18.
Vasconcelos, M., & Urcuioli, P. J. (2009).
Extensive training is insufficient to produce
the work-ethic effect in pigeons. Journal of
the Experimental Analysis of Behavior, 91,
143–152.
Vasconcelos, M., Urcuioli, P. J., & Lionello-
DeNolf, K. M. (2007). Failure to replicate
the ‘‘work ethic’’ effect in pigeons. Journal
of the Experimental Analysis of Behavior, 87,
383–399.
Vorndran, C. M., & Lerman, D. C. (2006).
Establishing and maintaining treatment
effects with less intrusive consequences via
a pairing procedure. Journal of Applied
behavior Analysis, 39, 35–48.
Waite, T. A., & Passino, K. M. (2006).
Paradoxical preferences when options are
identical. Behavioral Ecology and Sociobiol-
ogy, 59, 777–785.
Wardle, J., Herrera, M. L., Cooke, L., &
Gibson, E. L. (2003). Modifying children’s
food preferences: The effects of exposure
and reward on acceptance of an unfamiliar
vegetable. European Journal of Clinical
Nutrition, 57, 341–348.
Zentall, T. R. (2005). A within-trial contrast
effect and its implications for several so-
cial psychological phenomena. International
Journal of Comparative Psychology, 18,
273–297.
Zentall, T. R. (2008). Within-trial contrast:
When you see it and when you don’t.
Learning & Behavior, 36, 19–22.
192 JAMES N. MEINDL

APPENDIX

Initial Middle
Author Population component component
Alessandri, Phases 2–3 Phase 2 Preferred vs.
Darcheville, N 5 30 different stimuli nonpreferred effort
Delevoye-Turrell, undergraduate
and Zentall, 2008 students
Phase 3 10-s delay vs.
different stimuli nonpreferred effort
Alessandri, N 5 42 children Different Stimuli No delay vs. 8-s delay
Darcheville, and age 7–8 years
Zentall, 2008
Arantes and Experiment 1 Experiment 1 FR 1 vs. FR 20
Grace, 2008 N 5 10 pigeons identical stimuli

Experiment 2 Experiment 2 FR 1 vs. FR 20

N 5 12 pigeons identical stimuli
Armus, 2001 N 5 7 rats No initial component 5-g effort vs. 75-g effort
lever press
Aw, Holbrook, N 5 13 banded tetras No initial component Food deprived vs. prefed
Burt de Perera,
and Kacelnik,
2009
Aw, Vasconcelos, Experiment 1 Experiment 1 3-s delay vs. 18-s delay
and Kacelnik, N57 identical stimuli
2011 wild-caught starlings
Experiment 2 Experiment 2 3-s delay vs. 18-s delay
N 5 12 different stimuli
wild-caught starlings (randomly alternated)
Experiment 3 Experiment 3 4-flight effort vs. 12-flight
N 5 12 different stimuli effort vs. 24-flight effort
wild-caught starlings
Clement, Feltus, N 5 8 pigeons identical stimuli FR 1 vs. FR 20
Kaiser, and
Zentall, 2000
Clement and Experiment 1 Experiment 1 Differential effort vs.
Zentall, 2002 N 5 8 pigeons different stimuli anticipated differential
effort
Experiment 2 Experiment 2 Expected vs. unexpected
N 5 8 pigeons different stimuli food
Experiment 3 Experiment 3 Assessed positive and
N 5 16 pigeons different stimuli negative contrast
High-probability food
vs. low-probability food

DiGian, Friedrich, N 5 16 pigeons Identical stimuli 0-s delay vs. 6-s delay
and Zentall, Different stimuli
2004

Friedrich and Experiment 1 Identical stimuli FR 1 vs. FR 30

Zentall, 2004 N 5 12 pigeons
Experiment 2 Identical stimuli FR 1 vs. FR 30
N 5 6 pigeons Uncorrelated with specific
terminal component
Friedrich, Clement, N 5 8 pigeons Different stimuli Food vs. no food
and Zentall, 2005
 UNDERSTANDING PREFERENCE SHIFTS 193

APPENDIX
Extended
MC initial Terminal component TC initial
preference assessed? discrimination task? pref assessed? Effect found?
Yes Yes; simultaneous No Yes for S+

Yes Yes; simultaneous No Yes for S+

No Yes; simultaneous No Yes for S+

No for S2

No Yes; simultaneous or No No; dependent on initiating

successive Tested with event
initiating event
No Yes; simultaneous No No; dependent on initiating
event
No No; grape- or bacon- No No
flavored pellet delivered
No No; food delivered in No Yes
different arms of Y maze

No No; colored key No No

presented

No No; colored key No No

presented

No No; colored key No Yes

presented

No Yes; simultaneous No Yes

No Yes; simultaneous No Yes for S+

Yes for S2

No Yes; simultaneous No Yes for S+

Yes for S2
No Yes; simultaneous No Yes for S+ and S2 in
positive and negative
contrast gp
No for S+ and
S2 in negative contrast
group
No Yes; simultaneous No Yes for S+ in identical
stimuli group
No for S+ in different
stimuli group
No for S2 in both groups
No No; left or right feeder Yes Yes
available
No No; alternation Yes No
between left or right
feeders
No Yes; simultaneous No Yes for S+ Yes for S2
194 JAMES N. MEINDL

APPENDIX
Continued

Initial Middle
Author Population component component
Gipson, Miller, N 5 16 pigeons Identical stimuli FR 1 vs. FR 30 50% or
Alessandri, and 100% reinforcement
Zentall, 2009

Kacelnik and N 5 12 wild-caught No initial component 4-flight effort vs. 16-

Marsh, 2002 starlings flight effort
Klein, Bhatt, and N 5 32 undergrad Identical stimuli FR 1 vs. FR 20 or FR 30
Zentall, 2005 students
Marsh, Schuck- N 5 12 wild-caught No initial component Food deprived vs. prefed
Paim, and starlings
Kacelnik, 2004
O’Daly, Meyer, Experiment 1 Identical stimuli VI 100 s vs. VI 10 s
and Fantino, N 5 20 pigeons Different stimuli
2005
Experiment 2 Different stimuli VI 100 s vs. VI 10 s
N 5 8 pigeons
Pompilio and N 5 6 wild-caught No initial component Food deprived vs. prefed
Kacelnik, 2005 starlings and delay: FI 10, 12.5,
15, and 17.5 s
Pompilio, Kacelnik, N unknown No initial component Low nutritional state vs.
and Behmer, 2006 grasshoppers high nutritional state

Singer, Berry, Experiment 2 Different stimuli Preferred vs. non-

and Zentall, 2007 N 5 8 pigeons preferred schedule:
DRO or FI
Vasconcelos, Experiments 1–3, 5 Experiments 1–5 Experiments 1–4
Urcuioli, and N 5 8 pigeons Identical stimuli FR 1 vs. FR 20
Lionello-DeNolf, Experiment 4 Experiment 5 FR 1
2007 N 5 4 pigeons vs. FR 40
Experiment 6 Experiment 6 Experiment 6
N 5 16 pigeons Identical stimuli FR 1 vs. FR 0
Different stimuli
Vasconcelos and Experiment 1 No initial component High deprivation vs. low
Urcuioli, 2008 N 5 4 pigeons deprivation
Experiment 2 No initial component High deprivation vs. low
N 5 6 pigeons deprivation
Vasconcelos and Experiment 1 Different stimuli FR 1 vs. FR 30
Urcuioli, 2009 N 5 6 pigeons
Experiment 2 Different stimuli 4 runs vs. 16 runs be-
N 5 4 pigeons tween panels
Waite and N 5 11 semitame No initial component 60-cm (high effort) vs.
Passino, 2006 gray jays 1.9-cm (low cost)
distance
Note. This table summarizes research by subject type, initial, middle component (MC),
terminal component (TC), and effect. Effects were reported based on the individual article. If
researchers reported a statistical measure, that was used to determine effect. If researchers
reported individual subject data, that was used to determine effect.
 UNDERSTANDING PREFERENCE SHIFTS 195

APPENDIX Extended,
Continued

MC initial Terminal component TC initial

preference assessed? discrimination task? pref assessed? Effect found?
No Yes; simultaneous No Yes for S+ with 100%
reinforcement
No for S+
with 50% reinforcement
Yes for S2 for both 50%
and 100% reinforcement
No No; colored key No Yes for 10 of 12 starlings
presented
No Yes; simultaneous No Yes for S+
No for S2
No No No Yes
No No No Yes for 15 of 17 pigeons

No No No Yes for 5 of 8 pigeons

No No No Yes for state (deprived vs.

prefed)
No No; lemon grass or No Yes
peppermint odor
presented
Yes Yes; Simultaneous No Yes for S+
S2 untested

No Experiments 1–5 No No
Yes; Simultaneous
No Yes; simultaneous No No
No No No Yes

No Yes; simultaneous No No for S+

No for S2

No Yes; simultaneous No No for S+

No for S2

No Yes; simultaneous No No

No No; color-coded No No
foraging
opportunities