See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/232565287

A brief measure of children's behavior problems: The Behavior Rating Index for
Children.

Article  in  Measurement and Evaluation in Counseling and Development · July 1984

CITATIONS READS
15 4,529

1 author:

Arlene Rubin Stiffman

Washington University in St. Louis
76 PUBLICATIONS   2,960 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Arlene Rubin Stiffman on 17 May 2014.

The user has requested enhancement of the downloaded file.

A Brief Measure of Children's
Behavior Problems: The Behavior
Rating Index for Children
This article reports the evaluation of the Behavior Rtlting t ndexfor Children, a l Svitem summated
category partition scale {hat provides a prorheuc measure of children's behavior problems,
Professionals who work with children are
continually confronted with a need for
short, reliable. easy-to-administer mea-
sures that apply to a broad range of issues.
It is frequently critical to have informa-
tion about children's behavior from in-
dividuals who observe them in a variety
of environments. Potential respondents
include parents. teachers, therapists, group
leaders, and the children themselves. Fur-
thermore. many research evaluations,
particularly those in school and comrnu-
nity settings, demand data about the same
child over a long period, including im-
mediately before and after treatment and
at longitudinal posttreatment intervals.
Research of this type often requires the
Arlene R. Stiffma.n is Co-Din'ctoT, Center for Ado-
lescent Mental. Health. Utashington University. St.
Louis. John. Go Orme is a postdoctoral fcllo<!\ D4.'-
partmeut of Behavioral Sciences, L'nil.'enitJ or Chi-
cago_ Deborali A. Euans is a system.s analyst, St. Louis
County Governmenl. Ronald .1.. Feldman is Director ..
Center for Adolestrru .Heutaf Health, JrasJu°ngton
University. Phoebe it [(tent?) is a research assistant,
GAlN Program, Washillgton Universit», The prep-
aration of this article has been [unded b)' research
s=« MH350;3 fro",lhe National Institute oj Mental.
Health. Office of Prevention. The authors graJefully
acknowledge the sUppo>t of the NJlI,fH.
ARLENE R. STIFFMAN
JOHN G. ORME
DEBORAH A. EVANS
RONALD A. FELDMAN
PHOEBE A. KEENEY
evaluation of children of differing ages or
of the same child at different ages. In all
these situations, the evaluation of a child's
behavior must be valid and reliable.
This article describes the development
and evaluation of the Behavior Rating I ndex
[07- Children (BRIC), which is capable of
being used in all the above evaluative sit-
uations. The BRIe is essentially a pro-
thetic measure of childhood behavior. It
measures only the degree of a behavior
problem. as opposed to metathetic instru-
ments that measure the direction or type
of such problems (Hudson, 1982).
Although numerous and various ques-
tionnaires purport to measure children's
behavior problems (Arnold & Smeltzer
1974; Borgaua & Fanshel 1970; National
Institute of Mental Health. 1980; Quay,
1977; Spivak. Swift. & Prewitt, 1971), none
can be used by multiple respondents for
children of all ages. Although many of the
existing measures possess adequate reli-
ability and validity, practical problems limit
their use for field or clinical research re-
garding the amelioration or prevention of
behavior problems in children. Some of
the available instruments are appropriate
only for children of restricted age ranges;
thus they cannot be used for longitudinal
JULY 1984
83
,
I
studies or studies of children who differ
greatly in their respective ages. Most ques-
tionnaires cannot accommodate the col-
lection of comparable information from
multiple respondents-children. parents,
teachers, and therapists. Dissimilar instru-
menrs make it difficult to compare be-
havior from one setting to another.
Many brief instruments actually require
15-30 minutes to complete. Although this
length is short for detailed behavior pro-
files. it is unwieldy for frequent test-retest
evaluations, is highly intrusive in treat-
ment situations, and is time consuming
for evaluations of groups or classes of chil-
dren (a particularly critical issue for school
and community evaluations). Conse-
quently, there is a need for a brief, ac-
curate instrument that is appropriate for
evaluating children of widely differing
ages, that is available in comparable form
for multiple informants, and that. applies
to group and classroom settings.
The authors of this article needed such
an instrument for a program of preven-
tive mental health research. A review of
the available literature revealed no instru-
ments that met all the above criteria. Con-
sequently, the authors developed the BRIC
in a form for a respondent who is eval-
uating a single individual and in a form
for a respondent who is evaluating a group
or classroom of children (Figure 1).
THE BEHAVIOR RATING
INDEX FOR CHILDREN
The BRIC is a 13-item. summated cate-
gory partition scale (Stevens, 19681. Of
the 13 items, 3 randomly placed items
(items 1, 6, and 10) indicate a behavioral
strength of the respondent. They are not
calculated in the behavior problem score;
only the 10 problem-oriented items are
scored. The 3 strength-oriented items are
embedded in the BRIe so that the re-
spondents are not confronted solely with
a list of "undesirable" behaviors and to
MEASUREMENT AND EVALUATION IN COUNSELING
\
84
discern evidence of extreme responses on
the part of respondents, Each item is rated
according to a 5-point Liken-type system
(l = rarely or never: 2 = a little of the
time; 3 = some of the lime; 4 = a good
part of the time; 5 = most or all of the
time).
The behavioral problems that were se-
lected for inclusion in the BRIC represent
behavioral dimensions or factors that ap-
pear repeatedly in established behavioral
inventories (Achenbach. 1978-1979;
Achenbach & Edelbrock, 1978}, in the
DSM-III Classification System for Chil-
dren (American Psychiatric Association,
1980), and in other instruments, regard-
less of age and sex. The BRIC, however,
was designed simply to measure the de-
gree of problem behavior.
The ten items that denote problem be-
haviors are used to compute a summary
score for the BRIC. The total score for
the BRIC is computed by employing a for-
mula proposed by Hudson (1982):
S = (Y - lvi (100) -7 4N. This formula
provides a simple linear transformation
that results in a BRIC summary score that
has a potential range of 0-1 00, regardless
of the number of items completed. Higher
scores indicate more severe behavioral
problems.
METHOD
The BRIC was evaluated through its use
with more than 600 children who partici-
pated in a field experiment known as the
GAIN (Group Activities for Individual
Needs) Program. The participants included
both referred and nonreferred children.
The nonreferred children were regularly
enrolled members of the Jewish Commu-
nity Center Association. The referred chil-
dren were recruited as participants because
one or both of their parents were in treat-
ment for a clinically diagnosed mental ill-
ness. The available research literature clearly
demonstrates that their parents' illness places
AND DEVELOPMENT
 o

Group name Dete

Re.pondent
PI••• plata the most appropriate numoer in the bOK beside each child's name. The number refen to the frequency with which a given problem is
•• hlbited,
1 • rarelv or never
2 - • IIttlo of the time
3 • tome of the time
4 Q good part of the time
;10

6 • m05t or en of the time

--------
Child', neme
.. ._- I,
f--- ~-
2, 3, 4, 6, 6, 7,
I 8, 9, 10, 11, 12. 13. 14,
--f--
1. app",,, to be relaxed or hoppy

2. hide. hi. thoughts from oth.,.

3. says or does reol!y strange things

4. doe. not pay attentlcn when he should

-
- . ...
5. quits a job or t8ik without finis:hing it

6. gets along well with otho"

-
7, hits, puih8l. or huru others
-
8, gots along poorly with others

9, go•• very upset

-
---
10. compliments or helps others
- ----.
11. ,•• I. sick
.. ----_ .
12. cheat.

c: 13. I•••• temper

C
r-
-
-<
FIGURE 1

..
'"
0> The Behavior Rating Index for Childre" (BRie) Group Report Form (CoPVright © Stiffman. 1983)

,j
them at high risk for behavioral problems
(Anthony, 1978; Clausen & Huffine, 1979;
Rutter, 1966).
The children participated in weekly ac-
tivity groups that attempted to enhance
their social competencies, coping skills. and
task abilities. Over 60 groups of children
(each consisting of a group leader and ap-
proximately eight children) participated
during the 2 years of the program.
During the program, the BRIC was
completed by the participant children at
repeated intervals (four times a year). The
referred children's behavior also was eval-
uated both before and after group treat-
ment by their parents or guardians (n
= 183), by their teachers (n = 198), by
group leaders (n = 28), and by nonpar-
ticipant observers (n = 13). The resultant
scores then provided data to determine
the internal consistency reliability of the
BRIC.
At the same limes they completed the
BRIC, the parents and teachers of re-
ferred children (n = 306) also completed
the lengthier (and thus unwieldy for group
measurement) lI8-item Child Behavior
Checklist (CBC) (Achenbach, 1978-1979).
This permitted the assessment of concur-
rent validity. In addition, a sample (n = 24)
of the referred parents and children com-
pleted the entire intake interview on a 2-
week test-retest basis. This provided data
for assessing reliability by means of test-
retest correlations and intraelass correla-
tion coefficients. Furthermore, approxi-
mately half of the leaders and observers
completed the BRIC twice within I week
to provide data for determining test-retest
reliability coefficients and interrater cor-
relations for their forms.
RESULTS
Table I shows the response bias, validity.
internal consistency, repeated-measures
reliability, and inrerrater reliability that
were evaluated.
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT
86
Response Set and Bias
Neither response set nor nonresponse to
individual items presented a problem. For
only 6 of the 875 completed forms did
respondents complete fewer than eight of
the ten scored items. Only one respondent .
completed both the positivity and nega-
tivity worded items in the same extreme
fashion.
Validity
The BRIe's concurrent validity and con-
struct validity are adequate. ConcuTTenJ va-
lidity refers to the degree of correspondence
between a given measure and external cri-
teria (Cronbach, 1971). It was found that
the phi correlation between referred chil-
dren's scores on the BRIC, as reported by
parents, and children's treatment status,
as reported by parents, was .65 (p < .001).
Because not all the children with behav-
ioral problems were treated for such
problems and because all the children who
were in treatment did not necessarily dis-
play problems (some were in family ther-
apy because of parental problems), a
perfect correspondence between these two
variables was neither expected nor found.
Construct validity was assessed through
the relationship between the parents' re-
port of the referred children's behavior
on the BRIC and the parents' report of
this behavior on the CBC. The BRIe scores.
from parents demonstrated high positive
correlations with the CBC behavior prob-
lem scores (r = .76, P < .001). This cor-
relation is strong, especially because a small
number of test items tends to attenuate
reliability.
Internal Consistency
Internal consistency reliability was ex-
amined through coefficient alpha (Cron-
bach, 1951), a measure that is based on
all of the interirem correlations within an
instrument. Coefficient alpha was com-
puted separately for each group of re-
 ./

TABLE 1
Validities and Reliabilities of the BRIC for Parents, Teachers,
Group Leaders, Observers, MId Children

Validity Reliabili~.
Treatment Internal Test..•_
__
BRie nt stetus cac consist8ncv Crand Interreter
(Phil Ir) (Alpha) intracl_1 Ir)
Parents .65" .76" .81·· .71-.72" .20 with
children·
Teachers na na .86·· na na
Groups leaders n8 na .86*· .89-.92·· .51 with
observers"
Observers n8 na .so" .84-.89--
Children na na .60-.70" .50"

*p < .001.
•'p <.0001.
na = not assessed.

spondents. It was consistently higher for
the BRICs completed by adults than for
those completed by children. Alpha was
.81 for parents, .81 for group leaders, .80
for observers, and .86 for teachers. In
contrast, it was .60 for nonreferred chil-
dren and. 70 for referred children.
Therefore, the internal consistency reli-
ability of the BRICs that were completed
by children seems to be marginal while
the reliability of the BRICs completed by
adults is adequate (Guilford & Fruchter,
1978; Hudson, 1982).
Repeated-Measures Reliability
Repeated-measures reliability is fre-
quently considered critical to any pro-
gram that attempts to measure change
over time (Kerlinger, 1973). Each group
of respondents (except teachers) that
completed the BRIC was asked to do so
again on a test-retest basis. Group lead-
ers (n = 1I) and observers (n = 6) from
the second year's cohort participated in
two test-retest periods. They were asked
to rate the children in their groups (n
= 155 and n = 84, respectively) twice
within 1 week, both at the end of a 4-
week baseline period and at the end of
treatment (approximately 25 weeks later).
To obtain test-retest data on parents,
24 referred families returned within 1 or
2 weeks of the initial interview for a sec-
ond interview. These families were re-
cruited from the second year's cohort of
referred families. All referred families were
asked to participate. The first 24 families
who agreed to be reinterviewed com-
prised the sample. At the second inter-
view, the parents completed a second
BRIC. Test-retest data for the BRICs
completed by children were not collected
systematically. Because the referred chil-
dren completed the BRIC at intake and
at the end of baseline, these retests formed
the nexus of test-retest data for children's
BRICs.
Repeated-measures reliability from all
respondents consistently demonstrated a
level that indicates the BRIC is an appro-
priate measure of behavioral change. Test-
retest reliability for BRICs completed by
group leaders was .89 (J1 < .001) after they
had only 4 hours of contact with the chil-
dren. (The groups had met four times for

JULY 1984

87
I hour each time. Not all children at-
tended every session.)
The intraclass correlation coefficient, a
more stringent measure of repeated-mea-
sures reliability because it controls for di-
rectional drift (Bartko, 1976), was also .89.
Test-retest reliabilities for BRICs corn-
pleted by observers was .84 after less than
1 hour of total contact with the children
(four observation sessions of 15 to 20 min-
utes each). The intraclass correlation coef-
ficient for observers was .85 <.p < .000 I).
These repeated-measures reliabilities for
group leaders and observers, which were
measured immediately after baseline, in-
dicate that with minimal observation, the
BRIC yields reliable data. When the same
test-retest procedure for observers and
group leaders was completed at postlest,
the intradass correlation coefficients and
Pearson rs reached .92 for group leaders
and .89 for observers.
The repeated parent interview dem-
onstrated that the lest-retest reliability for
BRICs completed by parents was. 72
(P < .0001). The intraclass correlation
coefficient was .71. Considering that the
test-retest reliability of the much longer
CBC ranges from .82 to .90 (Achenbach,
1978-19i9), .71 seems to be adequate for
such a short instrument.
Unfortunately, systematic test-retest
comparisons of BRICs completed by
children were not obtained. Test-retest
reliability computed between intake and
baseline for the referred subjects (n
'" 180) and controlled for the time in-
terval was .50 (P < .0001). It should be
noted that the testing situations differed
markedly: one was a personal interview
and the other an anonymous group test-
ing situation. Furthermore, the reliabil-
ity did rise to .58 for those children who
were 12 years old or older, indicating
that older children tend to be more re-
liable reponers. It is clear that BRICs
completed by children are less reliable
than those completed by adults.
MEASUREMENT AND EVALUAnON IN COUNSEUNG AND DEVELOPMENT
B8
Interrater Reliability
Interrater reliability is computed to de-
termine whether two different respon-
dents, observing the same behavior, report
similar scores on an instrument. Observ-
ers and group leaders and parents and
children simultaneously completed their
versions of the BRIC.
Interrater reliabilities between BRICs
completed by observers and group leaders
and those completed by parents and chil-
dren were not nearly as high as the test-
retest or internal consistency reliabiliries. In
p.-.1l"t,
this is to be expected; different re-
spondenrs typically do not see children un-
der exactly the same circumstances, nor do
they evaluate behavior according to exactly
the same standards. The correlation be-
tween BRIC scores for group leaders and
nonparticipant observers was only .51
Cp < .001, n = 84). Such interrater relia-
bility seems adequate, especially when one
considers the brevity of the instrument and
the difference between the two respon-
dents in the form of contact and the length
of contact with the children. It certainly
compares well with interrater reliability for
oilier instruments. The CBC, which is al-
most ten limes longer, yields an interparent
correlation ranging only from .54 to .74
(Achenbach, 1978-1979).
Parents and children completed the BRlC
at the same time. It was expected that there
would not be extensive agreement between
the behavioral ratings by parents and chil-
dren; that expectation held true. The cor-
relation between parent and child reports
of the BRIC is only .20 (p < .001) for chil-
dren ages 6 to 15 (n = 306). Although it
changes to .28 (p < .001) when only chil-
dren over 12-years old are included, it is
certainly low. This suggests that the chil-
dren see their behavior quite differently than
do their parents.
Because none of the other respondents
completed the BRIC at the same time or
were evaluating behavior in similar situ-
[
oflarge groups of children prior to more
extensive metathetic testing of those chil-
dren who are evidencing problems.
With the exception of the children's
form, the BRIe generally demonstrates
satisfactory reliability and validity for di-
verse groups of respondents who are fa-
miliar with a child's behavior. This
instrument should be a valuable addition
to the diagnostic arsenals of clinicians,
teachers, group leaders, and researchers
who seek an instrument that possesses not
only satisfactory psychometric character-
istics but also offers case of administration
and appropriateness for a wide range of
individuals.
REFERENCES
Achenbach, TM. (1978-1979). The Child Be-
havior Profile: An empirically based sys-
re m for assessing children's behavior
problems and competencies. International
[ournal of Menial Health, 7, 24--42.
Achenbach. T.M., & Edelbrock, C.S. (978).
The classification of child psychopathol-
ogy: A review and analysis of empirical
efforts. 85, 1275-130 I.
P!»)'choftJ~urtl Bulletin.
American Psychiatric Association. (l980). Di-
agnostic mid statistical ma-nual uf mental dis-
orders (Srd ed.). Washington, DC: Author.
Anthony, E..J. (1978). From birth to break-
down: A prospective studv or vulnerabil-
ity, In EJ Anthony, C. Koupernik, & C.
Chiland (Eds.), The child ill his family: V!ll-
nerablechiidren (Vol. 4) (pp, 273-286). New
York: Wile,·.
Arnold. E.. & Smeltzer. D.]. (197-iJ. Behavior
checklist factor analysis for children and
adolescents .. +rchives of Genera! Vl.w:hialry.
30. 799-804.
Bartko. J.J. (1976). Oil various intraclass cor-
relation reliability coefficients. Psvchological
Bulletin, 83. 762-765.
MEASUREMENT AND EVALUATION
90
Borgaua. E.F .. & Fanshel, D. (J 970). The Child
Behavior Characteristics (CBe) form: Re-
vised age-specific forms. A:fullivm7ate Be-
iunsiorol Research, 5, 49-82.
Clausen, J.A .. & Huffine. c,L. (19791. The im-
pact ot parental mental illness on children.
Research ill Commuuitv Mental Health, J.
183-214. -
Cronbach, LJ. (195 I). Coefficient alpha and
the internal structure of tests. Psscheme-
trika. 16. 297-334.
Cronbach, LJ (197Ii. Tesl validation. In R.L
Thorndike (Ed.), Educational measurement
(2nd ed.) (pp , 443-507). Washington:
American Council on Education.
Guilford, ].P .. & Fruchter, B. (1978). Funda-
menial suuistics in psycholugy and eduauion (6th
ed.). New York: McGraw-Hill.
Hudson, W.W. (1982). The clinical measurement
packnge: it field manual. Homewood, IL:
Dorsey.
Kerlinger, F.N. (1973). Foundations of behaviora!
"search (2nd ed.). ;";e"· York: Holt, Rine-
hart and Winsron.
National Institute of Mental Health. Series AN
1'10. I. (1980). 71!e assessment of ps),chnpath-
ology muJ behaoioral "rob/e-ms in children: A
review rf scales suitable for epidemiological ami
clinical research (l96'7-J979i (DHHS Pub-
lication No. Am,1 80-H);l7). Washington,
DC: Government Printing Office.
Quay, H.C. n 977). Measuring dimensions of
deviant behavior: The Behavior Problem
Index. [oumal of Abnormal Child Psyclwlogy.
5,277-287.
Rutter. M. (1966). Children of sick parents: .4n
enoironmentol and psychiatric siud». London:
Maudsley Monograph 16.
Spivak, G., Swift, G., & Prewitt, J- (1971). Syn-
dromes of disturbed classroom behavior:
A behavioral diagnostic system for ele-
rnenrarv schools. Journal of Special Educa-
lion, 5. 269-292.
Stevens, SS (1968). Ratio scales of opinion. In
D.K. Whitla (Ed.). Hmldbook of measurement
and as..us..
~lJll.·111 in behninorai scinta (PI)' 171-
199). Reading, MA: Addison-Wesley.
IN COUNSELING AND DEVELOPMENT
 ations, the interrater correlations between
BRICs completed by parent and teacher.
by teacher and group leader. by teacher
and observer. and by parent and group
leader were not computed. Children's be-
havior is known to be situation specific.
Clinical Cutting Point
Frequently. clinicians or researchers who
use a brief prothetic screening device such
as the BRIC require some form of clinical
cutting point by which they can evaluate
not only the extent of a problem, but
whether or not the problem seems severe
enough to warrant further testing. diag-
nosis. or treatment. There are several
methods for developing such a cutting
point. All involve the use of established
measures. which can indicate whether or
not a child has a clinically significant be-
havior disorder. Because the CBC score
was available from parents' data. cumu-
lative frequencies were computed for both
the referred children's CBC and BRIC
scores to determine a cutting point for the
adult versions of the BRIC.
Using the CBC's clinical cutting point
of63 as a criterion. parent-reported BRICs
demonstrated a cutting point of 30. Only
16% of the children falling within the CBC
clinical range had BRIC scores lower than
30. Thus, using scores greater than 30 as
indicative of a clinical problem results in
a misclassification rate of only 18%. This
is well within the misclassification range
for other longer instruments.
Under no circumstances should a cli-
nician or researcher automatically assume
that children with scores above 30 have
clinical problems and those under 30 do
not. Scores above 30 can be considered to
indicate that more detailed metathetic
testing devices. which are designed to de-
termine specific diagnoses. should be used.
Thus the BRIe could effectively serve as
a preliminary screening device for large
numbers of children. Its use could vitiate
the need for unnecessarily detailed di-
View publication stats
"glioses when there is no need. More ex-
tensive and time-consuming testing could
he reserved for those children who seem
to warrant it on the basis of speedier pre-
liminary screening devices. such as the
BRIe.
DISCUSSION
The BRIe offers several practical advan-
tages for clinicians and researchers. It is
extremely brief and can be completed in
less than 3 minutes by adults or children.
Therefore. it can be administered easily
at repeated intervals over the course of
treatment and at follow-up intervals with-
out intruding unduly on treatment time.
It is especially appropriate for group test-
ing situations in which a large number of
children need to be evaluated expedi-
tiously, Also. it can be scored quickly and
easily by hand. Furthermore. BRIe scores
range from 0 to 100. regardless of the
number of items completed. Thus, the
range of possible scores is standardized
and readily comparable. The BRIC can
be used in parallel forms by many differ-
ent respondents, including parents,
teachers. therapists, and children. Finally,
the BRIC is appropriate for both male
and female children who are 7- to IS-years
old.
Analyses of response set. validity, reli-
ability. and clinical cutting points dem-
onstrate that the BRIC is a promising new
instrument that possesses appropriate
psychometric and practical characteristics
for practice and research. This is espe-
cially [rue for the BRIe forms that are
used by adults. Response set poses no
problems. Internal consistency reliability
~s adequate. Interrater reliability is mar-
ginal but adequate. Test-retest reliability
is high. Concurrent and construct validi-
ties are adequate. Furthermore. a clinical
cutting point of 30 results in an acceptable
misc1assification rate, so the instrument
seems appropriate for prothetic screening
JULY 1984
89