JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2017, 9999, n/a–n/a NUMBER 9999 ()

THE EFFECTS OF PREFERENCE ASSESSMENT TYPE ON PROBLEM

BEHAVIOR
SARA BETH TUNG
KENNEDY KRIEGER INSTITUTE AND UNIVERSITY OF MARYLAND, BALTIMORE COUNTY

JEANNE M. DONALDSON
LOUISIANA STATE UNIVERSITY

AND

SUNGWOO KAHNG
UNIVERSITY OF MISSOURI

Individuals with intellectual and developmental disabilities who engage in problem behavior
maintained by access to tangibles may exhibit more problem behavior during certain preference
assessments. We compared three common preference assessments to determine which resulted
in fewer problem behaviors. The paired stimulus and multiple-stimulus without replacement
assessments produced higher rates of problem behavior than the free operant (FO) assessment,
suggesting that the FO assessment may be the most appropriate assessment for individuals who
engage in problem behavior maintained by access to tangibles.
Key words: preference assessments, problem behavior, tangible function

Preference assessments are frequently admi-
nistered to individuals with intellectual and
developmental disabilities to empirically identify
stimuli that may serve as reinforcers. The most
common models of preference assessments
involve presenting potential reinforcers in pairs
(paired stimulus or PS) or larger arrays (multi-
ple-stimulus without replacement; MSWO),
Sara Beth Tung, Department of Behavioral Psychology,
Kennedy Krieger Institute; Jeanne M. Donaldson, Depart-
ment of Psychology, Louisiana State University; SungWoo
Kahng, Department of Health Psychology, University of
Missouri.
This research is based on a project submitted in partial
fulfillment of requirements for the masters in arts degree
by the first author. We thank Casey Allen and Kristi Jeff-
ery for their assistance with data collection. We thank
John Borrero and Natalie Rolider for comments on previ-
ous versions of this manuscript.
Sara Beth Tung is now at the Marcus Autism Center.
Correspondence concerning this article should be
addressed to SungWoo Kahng, Ph.D., Department of
Health Psychology, University of Missouri, 205 Portland
St., Columbia, MO 65211. E-mail: kahngs@health.
missouri.edu
doi: 10.1002/jaba.414
© 2017 Society for the Experimental Analysis of Behavior
1
allowing participants to select and engage with
or consume the selected stimulus for a period of
time, removing the stimulus, and then repre-
senting the next array (Hagopian, Long, &
Rush, 2004). An alternative format involves
providing free access to multiple potential rein-
forcers simultaneously and measuring partici-
pant engagement (a free operant [FO] format).
Despite their utility in identifying reinfor-
cers, the repeated removal of stimuli during a
PS or MSWO assessment may exacerbate prob-
lem behavior maintained by access to tangible
items (Roane, Vollmer, Ringdahl, & Marcus,
1998). However, this is less of a concern when
using the FO format, in that items are typically
not removed during the assessment. Kang
et al. (2010) found that two participants, whose
problem behavior was maintained by access to
tangible items, engaged in problem behavior
during the PS and MSWO assessments but not
during the FO assessment. Kang et al. (2011)
similarly found higher levels of problem behav-
ior during the PS and MSWO assessments
2 SARA BETH TUNG et al.

relative to the FO assessment for individuals boy diagnosed with autism and ID. We con-
with problem behavior maintained by access to ducted sessions in a padded room or in a bed-
tangible items. room on the living unit. Functional analyses
Although these studies shed light on how var- indicated that all participants’ problem behav-
iations in preference assessment formats may ior was maintained by access to tangible
affect problem behavior, there were several lim- items.
itations. Although FO assessments do not
involve removing items within the assessment, Measurement and Interobserver Agreement
the therapist typically removes all items at the
Observers collected data on the problem
conclusion of the assessment, which may also
behavior (including self-injurious behavior,
occasion problem behavior. Neither Kang
aggression, and disruption) frequency, item
et al. (2010) nor Kang et al. (2011) collected
selection, and item engagement duration. We
data on problem behavior following the FO
individualized measures of problem behavior
assessment, which should be considered in
across participants and reported these measures
selecting assessment formats. Also, Kang
as a rate. Item engagement included the partici-
et al. (2010, 2011) did not hold constant the
pant manipulating an item for 3 s or more for
access duration to selected items. Longer access
nonedible items or placing an edible past the
to a potential reinforcer during the FO assess-
plane of his or her lips.
ment may function as an abolishing operation;
A second observer simultaneously but inde-
thus, reducing the probability of problem behav-
pendently scored the occurrence of the depend-
ior when the item is removed (i.e., differences in
ent variables, and we compared observers’
problem behavior may result from reinforcer
records to calculate interobserver agreement
satiation rather than frequent removal).
(IOA). A second observer collected data on prob-
The purpose of this study was to compare
lem behavior for 34% of sessions. When com-
PS, MSWO, and FO preference-assessment
pared to the primary observer using the
formats to determine problem behavior rates
proportional agreement method, IOA between
both during the assessment and following the
observers averaged 99% (range, 98%-100%),
removal of all items at the end of each assess-
92% (range, 80%-100%), 99% (range, 95%-
ment. Further, we controlled for the item-
100%), and 98% (range, 96%-100%) for Noah,
access duration across the three assessments to
Brady, Sofie, and Mason, respectively. A second
clarify the role of stimulus removal in evoking
observer collected data on item engagement dur-
problem behavior.
ing 44% of sessions and when compared to the
primary observer using a trial-by-trial agreement
METHOD method, IOA averaged 99% (range, 97%-
100%), 99% (range, 98%-100%), 99% (range,
Participants and Setting
98%-100%), and 99% (range, 97%-100%) for
Four children admitted to an inpatient unit
Noah, Brady, Sofie, and Mason, respectively.
for the assessment and treatment of problem
behavior participated. Noah was an 8-year-old
boy diagnosed with autism, intellectual disabil- Procedure
ity (ID), bipolar disorder, and attention deficit We exposed participants to FO, PS, and
hyperactivity disorder. Brady was a 9-year-old MSWO assessment formats, alternated in a
boy diagnosed with autism and ID. Sofie was counterbalanced order in a multielement
an 8-year-old girl diagnosed with autism and design. Each assessment included six items,
mild ID. Finally, Mason was a 10-year-old which we kept constant across all preference
 PROBLEM BEHAVIOR DURING PREFERENCE ASSESSMENTS
assessments for each individual. These items
were formally nominated by caregivers using
the Reinforcer Assessment for Individuals with
Severe Disabilities (Fisher, Piazza, Bowman, &
Amari, 1996), and we included the item from
the tangible condition of the functional analysis
for each participant. We selected leisure items
for all participants except Noah whose assess-
ments included only edible items. We consid-
ered one complete preference assessment a
session, and we conducted one to three sessions
per day. Following the completion of each pref-
erence assessment, we continued collecting data
for an additional 30 s to capture problem
behavior when all assessment materials were
removed. We did not deliver any social conse-
quences for problem behavior during or follow-
ing the assessment.
FO preference assessment. To equate the dura-
tion of item access across assessments, we con-
ducted the FO assessment first in each series of
three sessions using procedures similar to those
described by Roane et al. (1998) except that
each FO session was 2 min. During the 2-min
assessment periods, we provided free access to
the six assessed items and measured engage-
ment with each item. For Noah, we provided
free access to six plates with ample edible items
for him to consume. From the FO assessment,
we identified the longest duration of engage-
ment with a single item and yoked access dura-
tion during the subsequent PS and MSWO
assessment within each series of three sessions.
For example, if the longest duration of item
engagement during an FO assessment session
was 1 min with the Barbie doll, then we pro-
vided 1-min access to each item selected
(including the Barbie doll) during the subse-
quent PS and MSWO assessments. We recalcu-
lated these durations following each FO
assessment, and these durations averaged 50 s
(range, 38 s to 68 s), 113.7 s (range, 110 s to
116 s), 93.7 s (range, 79 s to 109 s), and
100.3 s (range, 85 s to 116 s) for Noah, Brady,
Sofie, and Mason, respectively.
3
PS preference assessment. We conducted the
PS assessment using the procedures outlined by
Fisher et al. (1992). That is, we presented sti-
muli in pairs and participant approaches to one
of the items resulted in access to that item for
the designated amount of time (based on the
FO assessment). Each PS session included
15 trials such that every item was presented in
a pair with every other item once.
MSWO preference assessment. We conducted
the MSWO assessment using the procedures
outlined by DeLeon and Iwata (1996). That is,
we initially presented all six items in an array
and following approach to an item, the thera-
pist delivered the item for the duration deter-
mined by the FO assessment. The item was
then removed and the remaining items in the
array were re-presented. Each MSWO session
included six trials such that each item was
selected once or the session was terminated fol-
lowing 30 s of nonselection. This occurred
only during the first MSWO session for
Mason—he selected only four items.
RESULTS AND DISCUSSION
Figure 1 shows rates of problem behavior
during each preference assessment (left column
of panels). Noah (top row) engaged in low rates
of problem behavior during each assessment.
Brady (second row), Sofie (third row), and
Mason (bottom row) engaged in higher rates of
problem behavior during the PS and MSWO
assessments relative to the FO assessment (note
the large differences in y-axis scaling across par-
ticipants). These results are similar to those of
Kang et al. (2010, 2011) who showed increased
problem behavior during FO assessments.
We also evaluated problem behavior levels
following the completion of each assessment
and these data are displayed in the right col-
umn of Figure 1. Noah engaged in low rates of
problem behaviors after each assessment (top
row). Brady’s problem behavior was equally ele-
vated across all three assessment formats
4 SARA BETH TUNG et al.

2.5
3.0 During Preference Assessment 2.5 Following Item Removal
2.0 2.0 Noah
1.5
2.0 1.5
1.0 MSWO 1.0
0.5
1.0 0.5
0.0 0.0
-0.5
0.0 -0.5
11 4 4 7 7 10 10 1 4 7 10
FO
FO Brady
50
50
40 40
PS
Problem Behavior per Minute

30 30
20 20
10 10
0 0
-10 -10
1 4 7 1 4 7

20 Sofie
20
15
15
10 FO 10
FO
5 5
0 0
-5 1 4 7 -5 1 4 7

36 36 Mason
27 27
18 18
9 9
0 0

-9 1 4 7 -9 1 4 7

Sessions

Figure 1. Problem behavior per min during three types of preference assessments (left panel) and following the
removal of all items (right panel).

(second row). Sofie and Mason (third and bot-
tom row, respectively) engaged in problem
behavior following each type of preference
assessment as well. These data suggest that
although the FO assessment tends to minimize
problem behavior during assessment sessions,
these assessments may still be associated with
problem behavior at the conclusion of the ses-
sion at rates equal to those associated with
other assessment formats. The highest (albeit
variable) rates were observed, however, follow-
ing the PS and MSWO assessments relative to
the FO assessment.
The value of a preference assessment is deter-
mined by its ability not only to minimize prob-
lem behavior but also determine a preference
hierarchy. We present the reported preference
hierarchies determined by each assessment in
Table 1. There was a strong degree of corre-
spondence, particularly among the top ranked
 PROBLEM BEHAVIOR DURING PREFERENCE ASSESSMENTS 5

Table 1 materials. One limitation we raised with Kang

Preference Hierarchies Across Each Assessment Type for et al. (2010) was that decreased problem behav-
Each Participant ior may have resulted from stimulus satiation
Item
rather than stimulus restriction or removal.
Participant Rank FO PS MSWO However, we observed increased problem
Noah 1 Popcorn Popcorn Popcorn behavior during PS and MSWO assessments,
2 Doritos Doritos Doritos even when equating stimulus access time across
3 Chips Crackers Crackers
4 Crackers Chips Chips
assessments, suggesting that the relevant evoca-
5 Pretzels Pretzels Pretzels tive event in the PS and MSWO assessments is
6 Potato Potato Potato the removal of items and not necessarily related
Sticks Sticks Sticks
Brady 1 iPad iPad Ball to deprivation from particular items.
2 Badge Ball iPad These collective findings suggest that the FO
3 Ball Badge Badge
4 NS Blanket Blanket assessment may be the most appropriate prefer-
5 NS Piano V-Tech ence assessment for individuals who engage in
6 NS V-Tech Piano
Sofie 1 Play Doh Play Doh Play Doh problem behavior maintained, at least in part,
2 DVD DVD DVD by access to tangible items. This assessment
3 Coloring Coloring Puzzle
4 Magazine Magazine Coloring
was associated with the least overall level of
5 NS Puzzle Magazine problem behavior and resulted in similar prefer-
6 NS Keys Keys ence hierarchies as the PS and MSWO. How-
Mason 1 DVD DVD DVD
2 Car Car Car ever, it should be noted that functional analyses
3 NS Ball Legos are not always conducted prior to preference
4 NS Book Book
5 NS Legos Ball assessments. Therefore, implementers may need
6 NS Sensory Ball Sensory Ball to select preference assessment formats based
Note. NS = not selected upon caregiver report of behavioral function
prior to conducting a functional analysis (Kang
et al., 2011). It is also possible that observing
items in each of the preference assessments problem behavior during a preference assess-
across participants. However, it is worth noting ment may be an indicator that a tangible condi-
that the FO assessment failed to identify a tion should be included in a functional
complete hierarchy for Brady, Sofie, and Mason analysis.
as they never selected some items. Similar An alternative process for determining rein-
results from an FO assessment were reported forcers for individuals who engage in problem
when activities were evaluated for adults with behavior maintained, at least in part, by access
disabilities (Hanley, Iwata, Lindberg, & Con- to tangible items may be to use pictorial-based
ners, 2003). The PS and MSWO assessments (Heinicke et al., 2016) or video-based (Clark,
always resulted in complete hierarchies, partly Donaldson, & Kahng, 2015) preference assess-
because they required participants to encounter ments. These may evoke fewer problem beha-
each stimulus. viors because highly preferred items are not
The results of the current study are similar removed from the participant during these
to those reported by Kang et al. (2010) in that assessments. The utility of pictorial-based or
we observed lower rates of problem behavior video-based preference assessments for indivi-
during the FO assessments when compared to duals who engage in problem behavior main-
the PS and MSWO assessments, presumably tained by access to tangible items requires
because the FO assessment did not involve fre- evaluation and thus remains an important area
quent removal or restriction of tangible of future research.
6 SARA BETH TUNG et al.
Identifying the minimum amount of time
necessary to empirically determine reinforcers
would be highly valuable in future research as
well. FO assessments are typically conducted in
5-min or longer durations (Roane et al., 1998;
Rapp, Rojas, Colby-Dirksen, Swanson, & Mar-
vin, 2010); however, this study used 2-min
assessment sessions, which produced similar
results as the PS and MSWO assessments.
Thus, it is possible that FO assessments can be
potentially more efficient than those reported
previously. Future research should also deter-
mine if the FO assessment would produce the
same results (i.e., similar preference hierarchies,
low levels of problem behavior during assess-
ments, and problem behavior immediately fol-
lowing the assessment) for problem behavior
maintained by reinforcers other than tangible
items (e.g., attention and escape from
demands) and whether this assessment can be
used to identify alternative reinforcers to facili-
tate tolerance for delayed reinforcement when
reinforcers maintaining problem behavior are
unavailable (Austin & Tiger, 2015).
REFERENCES
Austin, J. E., & Tiger, J. H. (2015). Providing alternative
reinforcers to facilitate tolerance to delayed reinforce-
ment following functional communication training.
Journal of Applied Behavior Analysis, 48, 663-668.
doi:https://doi.org/10.1002/jaba.215
Clark, D. R., Donaldson, J. M., & Kahng, S. (2015). Are
video-based preference assessments without access to
selected stimuli effective? Journal of Applied Behavior
Analysis, 48, 895-900. doi:https://doi.org/10.1002/
jaba.246
DeLeon, I. G., & Iwata, B. A. (1996). Evaluation of a
multiple-stimulus presentation format for assessing
reinforcer preferences. Journal of Applied Behavior
Analysis, 29, 519-533. https://doi.org/10.1901%
2Fjaba.1996.29-519.
Fisher, W. W., Piazza, C. C., Bowman, L. G., &
Amari, A. (1996). Integrating caregiver report with a
systematic choice assessment. American Journal on
Mental Retardation, 101, 15-25.
Fisher, W., Piazza, C. C., Bowman, L. G.,
Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A
comparison of two approaches for identifying reinfor-
cers for persons with severe and profound disabilities.
Journal of Applied Behavior Analysis, 25, 491-498.
https://doi.org/10.1901%2Fjaba.1992.25-491
Hagopian, L. P., Long, E. S., & Rush, K. S. (2004). Pref-
erence assessment procedures for individuals with
developmental disabilities. Behavior Modification, 28,
668-677. https://doi.org/https://doi.org/10.1177/
0145445503259836
Hanley, G. P., Iwata, B. A., Lindberg, J., & Conners, J.
(2003). Response-restriction analysis: I. Assessment
of activity preferences. Journal of Applied Behavior
Analysis, 36, 47-58. https://doi.org/10.1901/jaba.
2003.36-47
Heinicke, M. R., Carr, J. E., Pence, S. T., Zias, D. R.,
Valentino, A. L., & Falligant, J. M. (2016). Assessing
the efficacy of pictorial preference assessments for
children with developmental disabilities. Journal of
Applied Behavior Analysis, 49, 848-868. https://doi.
org/10.1002/jaba.342
Kang, S., Lang, R. B., O’Reilly, M. F., Davis, T. N.,
Machalicek, W., Rispoli, M. J., & Chan J. M.
(2010). Problem behavior during preference assess-
ments: An empirical analysis and practical recommen-
dations. Journal of Applied Behavior Analysis, 43, 137-
141. https://doi.org/10.1901/jaba.2010.43-137
Kang, S., O’Reilly, M. F., Fragale, C. L., Aguilar, J. M.,
Rispoli, M., & Lang, R. (2011). Evaluation of the
rate of problem behavior maintained by different
reinforcers across preference assessments. Journal of
Applied Behavior Analysis, 44, 835-846. https://doi.
org/10.1901%2Fjaba.2011.44-835
Rapp, J. T., Rojas, N. C., Colby-Dirksen, A. M.,
Swanson, G. J., & Marvin, K. L. (2010). Predicting
preference for items during periods of extended access
based on early response allocation. Journal of Applied
Behavior Analysis, 43, 473-486. http://doi.org/https://
doi.org/10.1901/jaba.2010.43-473
Roane, H. S., Vollmer, T. R., Ringdahl, J. E., &
Marcus, B. A. (1998). Evaluation of a brief stimulus
preference assessment. Journal of Applied Behavior
Analysis, 31, 605-620. https://doi.org/10.1901%
2Fjaba.1998.31-605
Received March 28, 2016
Final acceptance November 1, 2016
Action Editor, Jeffrey Tiger