Journal of Autism and Developmental Disorders, Vol. 30, No.

4, 2000

The Children’s Social Behavior Questionnaire

for Milder Variants of PDD Problems: Evaluation
of the Psychometric Characteristics

Ellen Luteijn,1 Frans Luteijn,2 Sandy Jackson,3 Fred Volkmar,4 and Ruud Minderaa1

The Children’s Social Behavior Questionnaire (CSBQ) contains items referring to behavior
problems seen in children with milder variants of PDD. Data of large samples of children di-
agnosed as having high-functioning autism, PDDNOS, ADHD, and other child-psychiatric
disorders were gathered. Besides the CSBQ, parents completed the Autism Behavior Check-
list (ABC) and the Child Behavior Checklist (CBCL). The data provided the basis for scale
construction of the CSBQ, a comparison of the CSBQ scales with other instruments and a
comparison of groups on scores on the CSBQ. The 5 scales obtained referred to Acting-out
behaviors, Social Contact problems, Social Insight problems, Anxious/Rigid behaviors and
Stereotypical behaviors. Results show that the CSBQ has good psychometric qualities with
respect to both reliability and validity. A comparison of the different groups showed that sig-
nificant group differences were found on all scales. In general, the autism group received the
highest scores, followed by the PDDNOS group and the ADHD group. Exceptions were on
the Acting-out scale, where the ADHD group scored highest and on the Social Insight scale,
where no significant difference was found between the PDDNOS group and the ADHD group.
Implications of the results and suggestions for further research are dicussed.

KEY WORDS: Children’s Social Behavior Questionnaire; psychometric characteristics.

INTRODUCTION Childhood Disintegrative Disorder in the DSM-IV

The last decade has seen a considerable increase in
discussions on the classification and description of the
problem behaviors of children with a Pervasive Devel-
opmental Disorder (PDD). This has led to the inclusion
of new or refined categories of pervasive developmental
disorders such as Asperger Disorder, Rett Disorder, and
1
University Center of Child and Adolescent Psychiatry, Groningen,
The Netherlands.
2
University of Groningen, Department of Clinical Psychology, The
Netherlands.
3
University of Groningen, Department of Developmental Psychology,
The Netherlands.
4
Yale University, Child Study Center, New Haven, Connecticut.
317
(American Psychiatric Association [APA], and ICD-10
(World Health Organization [WHO], 1993) classification
systems. At one stage a whole range of conditions was
referred to as “the autistic continuum.” These conditions
were described as “the triad of impairments” (impair-
ments of reciprocal social interaction, verbal and non-
verbal communication, and imagination) (Wing & Gould,
1979), or involved subclassification of autistic disorders
on the basis of the quality of social interaction (Aloof,
Passive, and Active but odd) (Wing & Gould, 1979).
More recently, several authors have begun to use the term
“autistic spectrum” (e.g. Bolton et al., 1994; Castelloe &
Dawson, 1993; Prior et al., 1998; Szatmari, 1992; Wing,
1988, 1997) to describe all forms of PDDs. Common to
all of these descriptions is the notion that developmental
0162/3257/00/0800-0317$18.00/0 © 2000 Plenum Publishing Corporation
318
disorders can be considered as combinations of problems
in different domains which vary in the amount and sever-
ity of problems.
Besides the alterations in classification systems, de-
scriptions of new subgroups of PDDs have emerged. An
example is the “Multiple Complex Developmental Dis-
order” (MCDD) category (Cohen, Paul, & Volkmar,
1987; Cohen, Towbin, Mayes, & Volkmar, 1994; Tow-
bin, Dykens, Pearson, & Cohen, 1993). This category
includes not only social interaction problems but also
thought disorders and impairments in the regulation of
affective state. Van der Gaag et al. (1995) showed that
in comparison with autistic children, MCDD children
were less disturbed on social interaction, communica-
tion, and stereotyped behaviors, yet more impaired on
thought problems, anxiety, and aggression. It is still un-
clear whether the MCDD category can be regarded as a
new subcategory of PDDs and if so, how many children
currently described in the residual category PDDNOS
(Pervasive Developmental Disorder Not Otherwise Spec-
ified) can be categorized according to the diagnostic rules
for MCDD (Buitelaar & Van der Gaag, 1998).
Despite these attempts at further classification, much
research on PDD is still required. This is most clearly the
case perhaps where PDDNOS is concerned. Consider the
fact that the two leading diagnostic systems, DSM-IV
(APA, 1994) and ICD-10 (WHO, 1993), have conceptu-
ally identical definitions for Autistic Disorder (e.g., Rut-
ter, 1996; Volkmar et al., 1994). In contrast, despite its
close relationship to autism, the diagnostic terms and
guidelines that describe the PDDNOS are not the same
(Towbin, 1997). The ICD-10 includes a separate “Atyp-
ical Autism” category to describe individuals who fail to
meet age criteria and/or in cases where the disorder does
not fully meet all criteria for antism with regard to the
number or areas of abnormality. The residual category of
the PDDs is described in the ICD-10 by the term “PDD-
unspecified,” that is, disorders that fit the general de-
scription of PDD but where contradictory findings or a
lack of adequate information mean that the criteria for
any of the other PDD categories cannot be met. In the
DSM-IV, on the other hand, the PDDNOS category is
described as a combination of the PDD-unspecified cat-
egory and Atypical Autism (Towbin, 1997).
The PDDNOS category (or the combined Atypical
Autism and PDD-unspecified category) seems to be a
considerable and relevant clinical group. According to
Volkmar, Klin, and Cohen (1997), for instance, the
PDDNOS category includes a larger number of individ-
uals than those who are stringently defined as autistic.
This means that the present descriptions of the various
specific categories of PDDs exclude a large clinical
Luteijn, Jackson, Volkmar, and Minderaa
group of individuals who suffer from pervasive devel-
opmental symptoms but who do not meet the diagnostic
criteria for one of these specific categories. In the Nether-
lands, a study by Van der Gaag and Van den Ban (1993)
showed that one out of three children hospitalized for
child psychiatric problems was diagnosed as having a
milder form of autism. It is unclear if this large propor-
tion reflects the extent of the problem or the possible
misuse of the term for severely disturbed children (Van
der Gaag, 1997). According to Prior et al. (1998), clin-
icians are confused about how to diagnose the various
PDD conditions in a reliable and valid way. This prob-
lem is particularly evident where PDDNOS is concerned.
DSM-IV describes PDDNOS children as suffering from:
severe and pervasive impairments in the development
of reciprocal social interaction or verbal and non-ver-
bal communication skills, or when stereotyped behav-
ior, interests, and activities are present, but the criteria
are not met for a specific Pervasive Developmental Dis-
order, Schizophrenia, Schizotypal Personality Disorder,
or Avoidant Personality Disorder. For example, this cat-
egory includes “atypical autism”—presentations that do
not meet the criteria for Autistic Disorder because of
late age of onset, atypical symptomatology, or sub-
threshold symptomatology, or all of these.
The observation by Mayes, Volkmar, Hooks, and Cic-
chetti (1993) that research on PDDNOS is limited and
this definition is used inconsistently is still valid. The
vagueness of the definition leads to the existence of a
very heterogeneous clinical group, with considerable
quantitative and qualitative variations in symptoms.
These variations create a second problem, namely, the
unclear boundaries with other mental disorders, espe-
cially with other PDDs, such as attention-deficit hy-
peractivity disorder (ADHD) (e.g., Jensen, Larrieu, &
Mack, 1997; Luteijn, Jackson, Volkmar, & Minderaa,
1998). There are also unclear boundaries with the nor-
mal population. Since there are no positively formu-
lated criteria for the category, it remains unclear which
and how many symptoms should be included. A fur-
ther problem is that of comorbidity with other impair-
ments or disorders such as attention problems, hyper-
activity, language disorders, and motor-skill problems.
This serves to complicate the reliability and validity of
classification.
Given these problems in the classification and de-
scription of the PDDNOS category, it is important to ex-
amine the specific characteristics of children who have
been classified as PDDNOS according to the DSM-IV
classification system. Identification of those character-
istics that are consistently present should help to decide
which should actually be included in the PDDNOS
Children’s Social Behavior Questionnaire
category. As Rutter and Schopler (1992) have pointed
out, definitions of autistic-like conditions still require
more clarification. Such clarification might lead to a con-
crete and positively formulated description of the
PDDNOS category or of subgroups within this category.
A reliable and valid instrument which describes the
problem behaviors of children with a pervasive devel-
opmental disorder in several domains would be helpful
for research as well as for clinical assessment. Several
diagnostic instruments are available to systematically di-
agnose individuals with autism. Examples include the
Childhood Autism Rating Scale (CARS; Schopler, Re-
ichler, De Vellis, & Daly, 1980; Schopler, Reichler, &
Renner, 1988), the Autism Behavior Checklist (ABC;
Krug, Arick, & Almond, 1980), the Autism Diagnostic
Interview-Revised (ADI-R; Le Couteur et al., 1989), and
the Autism Diagnostic Observation Schedule (ADOS;
Lord et al., 1989). For children with PDDNOS no spe-
cific diagnostic instruments are available. No instrument
has been developed to describe lesser variants of prob-
lem behaviors in several important developmental do-
mains, such as social interaction, communication with
other individuals, and stereotyped behaviors. The use of
a standardized checklist (such as the CBCL; Achenbach,
1991), on a broad range of behavioral and emotional
problems, can be regarded as a first step towards de-
scribing the problems in the PDDNOS category. How-
ever, this instrument does not yield specific and exten-
sive information on PDDs.
These considerations led, in 1994, to the develop-
ment of the first version of the Children’s Social Be-
havior Questionnaire (CSBQ). The instrument was de-
signed to be completed by parents and caregivers of
children ages 4–18 and contains items on social inter-
action, communication, stereotyped behaviors, motor
behavior, attention, affect regulation, sensory abnor-
malities, and understanding social cues. In the light of
the results of an initial study with the CSBQ (Luteijn
et al. 1998), the instrument was refined by removing
irrelevant or unsatisfactory items.
The present study was designed to provide infor-
mation on the psychometric qualities of the adapted
version of the instrument (contents, reliability, and con-
current and discriminative validity) and to provide data
on the CSBQ in relation to other instruments.
METHOD
Sample
The sample consisted of five groups of children,
whose parents were approached in a variety of different
319
ways. Most were approached through an outpatient clinic
specializing in developmental disorders at the Depart-
ment of Child and Adolescent Psychiatry in Groningen,
in the Netherlands. Children were selected from a total
of 1,886 who were referred in the period January 1996
to May 1997. The first group comprised 240 children with
problems classified as PDDNOS in accordance with the
DSM-IV. These children had severe problems in social
interaction or communication, or a restricted repertoire
of activities and interests. None met the DSM-IV crite-
ria for Autism or Asperger Disorder. Children who also
met the criteria for ADHD were excluded from the sam-
ple. Comorbidity with other DSM-IV disorders appeared
in 12.5% of the 240 PDDNOS children. These disorders
included elimination disorders (enuresis and/or enco-
presis) (n = 12); tic disorders (n = 4); learning disorders
(n = 4); adjustment disorders (n = 2); mood disorders
(n = 2); oppositional defiant disorders (n = 2); post-
traumatic stress disorder (n = 2); and problems related to
abuse or neglect (n = 2).
The second group comprised 95 high-functioning
autistic children. This group consisted partly of chil-
dren who were referred to the outpatient clinic and were
diagnosed as having an Autistic Disorder without being
mentally retarded. Because of the scarcity of these chil-
dren, most of the group (96%) were obtained through
Autism Teams working in the different regions of the
Netherlands (the response rate of this group is un-
known, since the parents were recruited by the Autism
Teams in order to ensure the anonymity of parents and
children). Information on comorbidity with other dis-
orders was not available.
The third group comprised 181 children who were
referred to the outpatient clinic and were diagnosed as
having ADHD. These children had attention-deficits with
or without hyperactivity problems. None met the crite-
ria for PDDNOS. Comorbidity with other DSM-IV
disorders appeared in 18.8% of the children. These dis-
orders were tic disorders (n = 10); oppositional defiant
disorder (n = 6); elimination disorders (enuresis and/or
encopresis) (n = 6); generalized anxiety disorder (n = 4);
somatization disorder (n = 2); phase of life problems
(n = 2), conduct disorder (n = 2) and disruptive behav-
ior disorder NOS (n = 2).
The fourth group was included as a clinical con-
trol group and comprised 400 referred children with a
range of other child psychiatric problems. These in-
cluded mood disorders, anxiety disorders, tic disorders,
obsessive-compulsive disorders, learning disorders,
sleep disorders, enuresis, parent–child relational prob-
lems, physical child abuse, disruptive behavior dis-
orders, and gender identity disorders. Children with
320
developmental disorders or ADHD were excluded from
the clinical sample.
Mentally retarded children were excluded from all
clinical groups, because it is to be expected that a re-
tarded level influences the adaptive behavior of a child.
Subsequently, this would influence the scores on the
CSBQ. In the outpatient clinic, classification into clin-
ical groups was carried out by a child and adolescent
psychiatrist following extensive diagnostic procedures.
These included several clinical interviews in which par-
ents or caregivers were questioned about their child’s
present functioning in various developmental domains.
Parents were also asked to describe the developmental
history of their child. Play contacts between the psy-
chiatrist and the child provided additional information,
concerning matters such as the social interactional and
communication capacities of the child. In most cases,
psychological assessment was used to provide infor-
mation concerning the child’s cognitive functioning
(and possible discrepancies in cognitive profile) atten-
tion, concentration, memory, and imaginative power.
A fifth group was included as a normal control
group and consisted of 234 normally developing chil-
dren ages 4 to 12 years (M age 8.3, range 4–12 years).
These children were approached through randomly se-
lected elementary schools in the north of The Nether-
lands. The parents or caregivers of these children all
declared that their children had never been in contact
with psychological or psychiatric services and had not
suffered from severe behavioral or emotional problems.
The general characteristics of the groups are shown
in Table I. The normal controls were drawn from a nar-
rower age range (4–12 instead of 4–18) and were some-
what younger than the clinical groups. In contrast to the
clinical groups, the normal controls had approximately
the same number of boys and girls. The proportions of
boys and girls in the clinical groups differed significantly
(χ2, p <.001). In the clinical control group, the propor-
Table I. General Sample Characteristics
PDDNOS Autism
(n = 240) (n = 95)
M SD M SD
Mean age 8.8 3.4 9.7 4.1
Range 4–18 4–18
Gender [n (%)]
Boys 181 (75%) 81 (85%)
Girls 59 (25%) 14 (15%)
Luteijn, Jackson, Volkmar, and Minderaa
tions of boys and girls were markedly different from the
other groups. In the PDD groups and the ADHD group
the proportion of boys was significantly larger. The mean
ages of the different clinical groups were also signifi-
cantly different (ANOVA, F = 2, 61, p = .05).
Procedure
On their first visit to the Department of Child and
Adolescent Psychiatry, parents were asked to complete
a questionnaire concerning the main problems of their
child and the developmental history. These questions
were supplemented with several standard question-
naires, including the CSBQ, the ABC, and the CBCL.
This information was gathered before the extensive di-
agnostic procedures had started, in order to obtain
parental judgments that were not influenced by dis-
cussions with clinicians. After the child had been di-
agnosed and the parents had given permission to use
the data for research purposes, the parent questionnaires
were processed (response rate 90%).
Where possible the psychiatrists also completed a
checklist of DSM-IV criteria for PDD for all of the clin-
ical groups. This was done following the diagnostic
procedures. Completed checklists were obtained for
78% of the PDDNOS group, for 57% of the ADHD
group, for 4% of the Autism group, and for 20% of the
clinical controls.
THE CHILDREN’S SOCIAL BEHAVIOR
QUESTIONNAIRE
Reference has already been made to the main char-
acteristics of the CSBQ. Since it is a new instrument and
since the present article describes research using a mod-
ified version of that described in Luteijn et al. (1998), the
following sections provide a detailed account of the main
features of the instrument and its construction.
ADHD Clinical controls Normal controls
(n = 181) (n = 400) (n = 234)
M SD M SD M SD
9.4 3.5 9.5 3.5 8.3 2.7
4–18 4–18 4–12
163 (90%) 269 (67%) 105 (45%)
18 (10%) 131 (33%) 129 (55%)
Children’s Social Behavior Questionnaire 321

Item Construction somewhat arbitrary item arrangement showed that most

items seemed to fit more than one item-group and that
The CSBQ (Luteijn et al. 1998) is a 96-item ques-
several of the nine a priori item-groups had high inter-
tionnaire for parents or caregivers of children with PDD
correlations (up to 0.7). These results indicate that a high
problems. The items were formulated on the basis of lit-
score on one dimension is closely related to a high score
erature concerning such children, parental descriptions,
on another. This means that the domains cannot be seen
and insights developed at the University Centre for Child
as independent and that they do not differ greatly from
and Adolescent Psychiatry in Groningen. The main aim
each other.
was to develop a checklist of the problem behaviors seen
In view of these findings, the structure of the CSBQ
in children with PDD. These behaviors have proved dif-
was examined by means of factor analyses. The struc-
ficult to assess when using the more general psycholog-
ture had to fit all groups as much as possible. A series
ical checklists which have been frequently applied in the
of principal component analyses on all items in the clin-
assessment of PDDNOS. The instrument offers the op-
ical groups was conducted to determine what items
portunity to describe a broad range of severe and less se-
should be retained. It was decided that the components
vere PDD features. The parent is asked to react to each
should include several items with high factor loadings,
item by indicating whether “it does not describe the
that they should explain a reasonable amount of vari-
child ” (score 0), “infrequently describes the child”
ance and, above all, that they should be interpretable.
(score 1), or “clearly applies to the child” (score 2),
For practical reasons it was decided that there should
when referring to the behavior of the child during the
not be more than 10 factors. The results indicated that
preceding 2 months. A first study, using a 135-item ver-
a five-factor solution best fitted the data. These five fac-
sion of the CSBQ, showed that the interrater reliability
tors were rotated to a varimax solution which accounted
between parents was satisfactory (Pearson r, range
for 39% of the total variance. Examination of the struc-
.47–.87, Mdn .79) and that the questionnaire seemed to
ture in all groups, including the normal control group,
have the potential to discriminate between children with
revealed that this five-factor solution explained ade-
PDDNOS and normally developing children (Luteijn et
quately the variance in all groups (normal control group:
al., 1998). In the light of the first study, the 135 items
29%, high functioning autism group: 35%, ADHD
were reduced to 96 items. This was done for three rea-
group: 37%, PDDNOS group 37%, clinical control
sons. First, some items appeared to apply less specifi-
group: 38%). The five factors were described as Act-
cally to the description of PDDNOS problems (e.g.,
ing-out, Social Contact Problems, “Social Insight Prob-
items describing basic regulation problems: eating and
lems,” Anxious/Rigid, and Stereotypical.
sleep problems). Second, a small number of items
showed minor variations in scores and were therefore
unfit for inclusion in the questionnaire. Finally, parents The CSBQ scales
indicated that they had difficulty in responding to cer-
On the basis of the five factors, items were divided
tain items because they were unclearly formulated. The
into five subscales. Items had to load considerably
new 96-item version was used in the present study.
(above .30) on a factor. To achieve further scale-inde-
pendence, items were only included in the scales if their
loadings on a particular factor differed by at least .1
The Structure of the CSBQ
from loading on another factor. Items that did not meet
The aim of examining the structure of the CSBQ this criterion were excluded from the scales. Applica-
was to identify, insofar as possible, independent compo- tion of these criteria meant that 66 items were included
nents in order to form scales. Ideally, these scales would in the scales. The majority of the 30 excluded items
describe autism-related problems in a broad child psy- showed high correlations with several scales and might
chiatric population by means of different domains, that therefore be rather important for describing (PDD-)
is, they would create a (PDD) problem profile. Given this problems.
aim, the data from the normal control group were ex- Table II presents the items of the five scales in
cluded from the calculations. At first, an a priori arrange- order of the loadings on the components (the highest
ment of the items into nine item-groups (based on the first). The eigen values of the rotated factors and the
first study with the CSBQ, Luteijn et al., 1998) was ex- percentages of explained variance of each of the fac-
amined by means of a simultaneous components analy- tors is also shown. It was decided to study these five
sis (SCA; Kiers, 1990). The results of this initial and scales in the present research. In general, the scales are
322
Table II. Items of the Five Scales of the CSBQ
Scale 1: Acting out (Cronbach’s α = .92; eigen value of the
factor: 9.1; % of variance: 9.5)
Is disobedient
Does not know when to stop, e.g., goes on and on about things
Cannot be corrected in situations in which he/she has done some-
thing wrong
Draws excessive attention to him/herself
Is extremely stubborn
Behaves aggressively
Quickly gets angry
Over-reacts to everything and everyone
Acts like “a clown,” e.g., in front of visitors
Is overactive, runs and flits to and fro
Behaves inappropriately in public places
Does not understand that certain things are “not done”
Cannot sit still; some part or other of him/her is always moving
Talks too loudly
Scale 2: Social Contact Problems (Cronbach’s α = .88; eigen
value of the factor: 8.1; % of variance: 8.5)
Has little or no need for contact with others
Does not begin to play with other children
Has difficulties associating with peers
Has few or no real friends
Does not respond to initiatives by others, e.g., does not play along
when asked to
Does not bother to keep a conversation going
Does not show his/her feelings in facial expressions and/or bodily
posture
Lives in a world of his/her own
Cannot imitate other people’s behavior; cannot “pretend to be”
Does not seek comfort when he/she is hurt or upset
Dislikes physical contact, e.g., does not want to be touched or
hugged
Acts as if others are not there
Discards things from the past too easily, e.g., major events have
not touched him/her
Scale 3: Social Insight Problems (Cronbach’s α = .91; eigen value
of the factor: 7.9; % of variance: 8.3)
Takes in information with difficulty
Talks confusedly; jumps from one subject to another in speaking
Does not fully understand what is being said to him/her, i.e., tends
to miss the point
Has difficulties doing two things simultaneously, e.g., he/she can-
not dress and listen to father or mother at the same time
readily interpretable and contain rather homogeneous
groups of items. The Acting-out scale contains exter-
nalizing, disruptive, and hyperactive behaviors. The
items describe problems in adapting behavior to the so-
cial context in an appropriate way.
The Social Contact Problems scale refers to a lack
of social interactions and a restricted intention to in-
teract. The items describe contact problems, both in
Luteijn, Jackson, Volkmar, and Minderaa
Does things without realising what stage of the activity he/she has
reached (beginning, middle, ending)
Does things without realizing the aim, e.g., constantly has to be re-
minded to finish something
Has difficulties in concentrating, e.g., on games
Has difficulties in concentrating on something for more than a
short period of time
Has no sense of time
Is exceptionally naive; believes anything you say
Pronounces words unclearly
Takes things literally, e.g., does not understand certain expressions
Does not understand jokes
Frequently says things which are not relevant to the conversation
Is clumsy in very fine work, e.g., buttoning up clothes
Has difficulties finding the way or the exit in other surroundings
Scale 4: Anxious/Rigid (Cronbach’s α = .86; eigen value of the
factor: 7.3 % of variance: 7.6)
Is unable to get certain things out of his/her mind
Panics in new situations or if change occurs
Is afraid to be separated from father/mother
Is overconcerned that something might happen to father/mother
Compared to peers, is particularly afraid of certain animals or situ-
ations
Has strange or bizarre thoughts
Gets worried about things, long before it is necessary to be so
Opposes change
Cries for incomprehensible reasons
Remains clammed up in new situations or if change occurs
Talks over and over again about something that happened in the past
Makes a point of doing certain things in the same way all the time
Reacts in an excessively scared or jumpy fashion to loud noises
Is excessively precise
Is oversensitive to pain
Beats, bites or scratches him/herself
Scale 5: Stereotypical (Cronbach’s α = .76; eigen value of the
factor: 4.8; % of variance: 5.0)
Flaps arms/hands when excited
Makes odd, fast movements with fingers or hands
Is fascinated by certain colors, forms, or moving objects
Constantly feels objects
Literally repeats words or sentences that have (just) been used by
someone else
Stands too close to strangers when talking to them
Is extremely pleased by certain movements and keeps doing them,
e.g., turning round and round
responding to approaches made by others as well as
in the intention to interact. The Social Insight Prob-
lems scale contains items that involve difficulties in
social orientation and picking up social cues. It in-
cludes some items that appear, at first sight, to be re-
lated to attention deficits. For example, “Talks con-
fusedly; jumps from one subject to another in
speaking” might refer to an attention deficit, but may
Children’s Social Behavior Questionnaire
also refer to a deficit in empathizing with other peo-
ple’s ideas. The Anxious/ Rigid scale includes items
that describe anxiety and resistance to change. It also
describes preoccupation with certain events which
happened in the past or will take place in the (near)
future. Finally, items in the Stereotypical scale, de-
scribe stereotyped behaviors as well as preoccupation
with sensory stimuli.
Other Instruments
The Autism Behavior Checklist. The ABC (Krug
et al., 1980) is a rating scale which is applicable to a
broad age spectrum—18 months to 35 years. It con-
sists of 57 behaviors measuring five dimensions (sen-
sory, relating, body and object use, language, and
social and self-help). It was developed as a screening
instrument to identify individuals with high levels of
autistic behavior in severely handicapped populations.
Some studies (e.g., Krug et al, 1980) have shown it to
be reliable (interrater reliability of .95 and intrarater
reliability of .87) and to have content, concurrent, and
criterion-related validity. Other research (e.g., Volk-
mar et al., 1988) has questioned the diagnostic value
of the ABC and has suggested that the instrument is
not suitable for establishing the diagnosis of autism.
This research pointed out, however, that the ABC ap-
peared to have merit as a screening instrument. The
ABC is used as such in the present study. The Total
score of the ABC is also used as a measure of autistic
behaviors.
The Child Behavior Checklist. The CBCL (Achen-
bach, 1991) is used to obtain standardized parental re-
ports of children’s problem behaviors of the preceding
6 months. The items of the questionnaire can be divided
into eight syndrome scales: social contact problems,
somatic complaints, anxious/depressed, delinquent
behavior, aggressive behavior, thought problems, so-
cial problems, and attention problems. Extensive re-
search with the Dutch version has shown that the CBCL
is reliable and valid and has provided normative data
(Verhulst, Berden, & Sanders-Woudstra, 1985; Ver-
hulst, Van der Ende, & Koot., 1996). In the present
study, the Total score on the CBCL is used as a mea-
sure of general psychopathology.
The Checklist of DSM-IV Criteria for PDD. This
is an instrument for clinicians, designed to establish the
presence of criteria for the Autistic Disorder and the
extent of their applicability to the PDDNOS group and
to the other clinical groups included in this study. The
checklist consists of the literal text of the DSM-IV.
Each item consists of one symptom and this meant that
323
sometimes a DSM-IV criterion was divided into sev-
eral concrete items. Each of the 20 items could be rated
as 0 (absent), 1 (mild), 2 (moderate), or 3 (severe). This
meant that less severe autistic symptoms could also be
rated. Eight items concerned problems in social inter-
action, seven referred to problems in (non)verbal com-
munication and five focused on stereotyped and re-
stricted activities or interests. Three additional questions
dealt with age of onset of problems in social interac-
tion, language, and pretend play. A small interrater
study (n = 15) showed that the checklist appeared to
have a high level of interrater reliability for the total
score intraclass correlation coefficient (ICC), .91
(Bartko, 1976), and for the Total scores for different
domains (Social Interaction, ICC = .82; Communica-
tion, ICC = .91; Restricted Repertoire of Activities and
Interests, ICC = .66). The internal reliability of the
scales was also satisfactory. When these data were ap-
plied to the group of children in the present study, the
alpha scores (Cronbach, 1951) were Social Interaction
a = .92 (n = 369); Communication a = .84 (n = 361),
and Restricted Repertoire of Activities and Interests
a = .83 (n = 371).
Statistical Analyses
Several analyses were used to examine the psy-
chometric qualities of the CSBQ. These consisted of
calculations of interrater reliability (between parents of
the same child) (ICC; Bartko, 1976); test–retest relia-
bility (of scores on the CSBQ within a few weeks) (ICC);
internal consistency of the scales (Cronbach’s α). The
intercorrelations (Pearson r) of the scales were also
calculated.
Correlations with other instruments (Pearson r)
were used to provide information on the validity of the
instrument. Next, the means and standard deviations of
the scores on the CSBQ scales in the different groups
were compared by means of an ANOVA. In case of a
significant ANOVA a post-hoc analysis was carried out
by means of a t test. Finally, a discriminant analysis
was carried out to determine the proportion of children
that the instrument could correctly classify.
The normal control group was included as a ref-
erence point for the scores of the clinical groups and
proved to have low variance and significantly lower
scores than the clinical groups on every questionnaire.
The results for this group were therefore not included
in the study of the psychometric qualities of the
CSBQ. However, they were included in the compari-
son of the scores of the different groups on the CSBQ
scales.
324
RESULTS
A Study of Reliability of the CSBQ
Interrater Reliability
For a first study of the interrater reliability of the
96-item version a small group of mothers and fathers
completed the CSBQ independently of each other (n =
23). A satisfactory level of interrater reliability was ob-
tained for the CSBQ Total Score (ICC = .83), the Act-
ing-out scale (ICC = .75), the Social Contact Problems
scale (ICC = .85), the Social Insight Problems scale
(ICC = .73), the Anxious/Rigid scale (ICC = .64), and
the Stereotypical scale (ICC = .72).
Test–retest Reliability
To examine the test–retest reliability, 21 mothers
were asked to complete the CSBQ for the second time
after an interval of approximately 4 weeks. Agreement
between the first and the second measurement was, with
the exception the Stereotypical scale, for most scales
satisfactorily high [CSBQ Total score ICC = .90, Act-
ing-out scale (ICC = .85), the Social Contact Problems
scale (ICC = .87), the Social Insight Problems scale
(ICC = .62), the Anxious/Rigid scale (ICC = .85), and
the Stereotypical scale (ICC = .32)].
Internal Consistency of the Scales
The internal consistency of the scales was satisfac-
torily high: Cronbach’s α: Scale 1 (14 items): .92; Scale
2 (13 items): .88; Scale 3 (16 items): .91; Scale 4 (16
items): .86 and Scale 5 (7 items): .76). (See Table II.)
Intercorrelations of the CSBQ Scales
Table III shows a correlation matrix of the five
scales of the CSBQ. The correlations vary from .30 to
.58. Since it was designed for children with pervasive
problems, the items of the CSBQ describe problems in
Table III. Overview of the Intercorrelations of the CSBQ Scalesa
Factor 2 3 4 5
1. Acting-out .30 .58 .32
2. Social Contact Problems — .46 .40
3. Social Insight Problems — .40
4. Anxious/Rigid — .52
5. Stereotypical
a
All correlations are significant (p <.01).
Luteijn, Jackson, Volkmar, and Minderaa
different developmental domains. Since items in each
scale refer to specific problems and children with de-
velopmental disorders have problems in more than one
domain, the presence of some significant correlations
is not unexpected.
The CSBQ in Comparison with the Other Parent
Questionnaires
The scales of the CSBQ were compared with the
scales of the CBCL and the ABC. It was expected that
the five scales of the CSBQ would correlate with the
scales of the other instruments on comparable topics.
The sum-scores of all instruments were also compared.
It was expected that the sum-scores of the CSBQ would
have high correlations with the sum-scores of the ABC,
as a measure of autistic behaviors, as well as with the
CBCL as a measure of general psychopathology. The
results are presented in Table IV.
The high correlations of the different scales of the
CSBQ with the scales of the standardized instruments
that cover comparable developmental areas indicate
that the scales of the CSBQ are valid. The first scale
of the CSBQ, Acting-out, has a very high correlation
with the CBCL Aggressive Behaviors scale. The items
on these scales are similar in content. Note that in the
CSBQ, as in the CBCL and the ABC, the title of a scale
does not always match the full content of the scale. For
instance, the Aggressive scale of the CBCL contains a
number of items that describing disruptive behaviors
that are not purely aggressive or arising from aggres-
siveness. Some items describe (or may be the result of)
PDD problems (e.g., Stubborn Behaviors) may be re-
lated to rigid patterns of thinking and not necessarily
to aggressive behaviors.
The second scale, Social Contact Problems, cor-
related most with the ABC “Relating” scale and the
CBCL Withdrawn scale. As with the CSBQ Social
Contact Problems scale, these scales contain items re-
ferring to a lack of relations or contacts with other
people, or to living in a world of one’s own, or not
wishing to have physical contact with another person.
The scores of the third scale of the CSBQ, Social
Insight Problems, correlated most with the Attention
scale of the CBCL. The Social Insight Problems scale
of the CSBQ contains items describing attention prob-
.46 lems and not automatically recognizing social cues.
.41 The fourth scale of the CSBQ, Anxious/Rigid, cor-
.52
related most with the CBCL Thought Problems scale
— and the CBCL Anxiety/Depression scale. The high cor-
relations with these two scales might be explained by
the fact that the CSBQ Anxious/Rigid scale contains
Children’s Social Behavior Questionnaire 325

Table IV. Correlation Matrix of the CSBQ Scales with the CBCL and the ABC Scalesa

CSBQ

Social Contact Social Insight

Variable Acting-out Problems Problems Anxious/Rigid Stereotypical

CBCL
Withdrawn .63 .40
Somatic Complaints
Anxiety/Depression .56
Social Problems .43 .55 .51 .33 .42
Thought Problems .35 .38 .56 .39
Attention .52 .37 .71 .36 .48
Delinquent .54 .35
Aggressive Behaviors .85 .48 .35

ABC
Sensory .32 .46 .45 .39 .47
Relating .41 .63 .48 .34 .41
Body/Object Use .47 .33 .45 .38 .61
Language .34 .44 .53 .54
Social & Self-help .60 .46 .56 .44 .52

a
Only correlations greater than .30 are shown.

many items that describe preoccupations concerning
situations or things. Fears and preoccupations might
have a strong relationship with thought problems.
The fifth scale of the CSBQ, Stereotypical, had the
strongest correlation with the ABC Body/Object Use.
Both scales contain stereotyped behaviors such as
stereotyped movements of the body.
The CSBQ in Comparison with the Clinician
Questionnaires
Clinical diagnosis was used to divide children into
different clinical groups. To support these clinical di-
agnoses and to describe accurately children’s problems,
especially those of children with PDDNOS problems,
the clinicians completed a checklist of the DSM-IV cri-
teria for PDD. This meant that it was possible to iden-
tify how many criteria of the Autistic Disorder were
applicable to the different groups. It also made it pos-
sible to compare parental judgment of the child’s be-
havior (in the CSBQ) with clinicians’ impressions.
Table V summarizes the correlations of the CSBQ
scales and Total scores with the DSM-IV questionnaire
scales and Total scores Clinicians’ scores on the check-
list of the DSM-IV criteria for PDD corresponded with
the parental scores on the CSBQ. Comparison of the
scores shows that to a certain extent the CSBQ measures
the same problems. This is shown by the high correla-
tion between the Total scores of both instruments and,
for instance, the sum of the social interaction problems
on the DSM-IV checklist and the Social Contact Prob-
lems scale of the CSBO. With the exception of Acting-
out and Anxious/Rigid, the CSBQ scales correlate mainly
with all three categories of the DSM-IV checklist. This
may indicate that these two scales are less specific to the
PDD problems described by the DSM-IV criteria.
Comparison of Different Groups by Means of the CSBQ
Since the CSBQ emerges as a reliable and valid in-
strument for describing the behaviors of PDDNOS chil-
dren on different dimensions, it is of interest to compare
the scores of the different clinical groups and the control
groups on the scales and the Total scores of the CSBQ.
Table VI shows the scores for all clinical groups and for
the Normal control group. When testing the differences
among the groups, only the Autism group, the PDDNOS
group, and the ADHD group were included. The clini-
cal control group was excluded, because it consisted of
children with a broad range of problems. Comparison
with this group would therefore not add any information
concerning their specific problems, which are likely to
vary considerably from one diagnosis to another. Simi-
larly, the Normal control group was not included, be-
cause it received, as expected, significantly lower scores
on all scales in comparison with the clinical groups.
An ANOVA of the five scales showed that the
scores of the groups on all scales with the exception of
Scale 3 (Social Insight Problems) differed significantly
( p <.001). Also the scores on Scale 3 differed, although
at a less significant level ( p = .01).
326 Luteijn, Jackson, Volkmar, and Minderaa

Table V. Correlations of the CSBQ scales and Total score with the DSM-IV Questionnaire
Scales and Total Score for Cliniciansa

DSM-IV

CSBQ Social Interaction Communication Stereotyped Total

Acting-out .46 .31 .40

Social Contact Problems .34 .35 .37
Social Insight Problems
Anxious/Rigid .37 .31
Stereotypical .44 .34 .32 .40
Total score

a
Only correlations greater than .30 are shown. These are all significant ( p <.01).

Table VI. Overview of Results of Comparisons of Clinical Groups and Normal Controls on the Scales of the CSBQ

PDDNOS Autism ADHD Clinical controls Normal controls

(n = 240) (n = 95) (n = 181) (n = 400) (n = 234)

Behavior M SD M SD M SD M SD M SD Fa Comparisonsb

Acting-out 15.0 7.0 14.3 5.7 17.4 6.3 11.4 7.6 4.8 4.4 9.0 ADHD> P, A
Social Contact 9.2 5.2 13.4 5.2 5.1 4.4 5.9 5.3 1.5 2.1 90.8 A> P > ADHD
Social Insight 14.8 7.5 16.8 6.5 14.1 6.7 9.1 7.4 3.7 4.0 4.5 A> P, ADHD
Anxious/Rigid 9.4 6.3 13.9 6.6 6.5 5.5 7.8 5.9 3.6 3.7 44.4 A> P > ADHD
Stereotypical 3.0 3.0 4.9 3.4 2.2 2.9 1.9 2.5 0.7 1.1 23.0 A> P > ADHD

a
Only the Autism group, the PDDNOS group, and the ADHD group were included in the ANOVA.
b
The symbols used express the differences between the groups (p <.05). A = Autism group; P = PDDNOS group; ADHD = ADHD group.

The following step was to compare the groups by
means of t tests. (Table VI gives an overview of the re-
sults). In Scale 1, which describes Acting-out behaviors,
the ADHD group received a significantly higher ( p <.01)
score in comparison with the other groups. Given the
contents of the scale, this is hardly a surprising result,
since the items describe many behaviors involving at-
tention problems. On the other hand, the high scores ob-
tained by the Autism group and the PDDNOS group were
unexpected, because acting-out behaviors are not nor-
mally included in the descriptions of these disorders. The
scores on the Acting-out scale of the Autism group and
the PDDNOS group did not differ significantly ( p = .4).
On the second scale, Social Contact Problems, the
Autism group had the highest scores, followed by the
PDDNOS group. In this case, the ADHD group obtained
significantly lower scores in comparison with both the
Autism group and the PDDNOS group ( p <.001).
The Autism group received again significantly
higher scores on the third scale, Social Insight Prob-
lems, in comparison with both the PDDNOS group and
the ADHD group ( p <.05). However, the scores of the
ADHD group did not differ significantly from those of
the PDDNOS group ( p = .39).
On the fourth scale, Anxious/Rigid, the Autism
group again received significantly higher scores in com-
parison with both clinical groups ( p <.001). The
PDDNOS group received significantly higher scores on
this scale in comparison with the ADHD group (p <.001).
Finally, the items of the fifth scale, Stereotypical,
appeared to fit the Autism group best, followed by the
PDDNOS group and then the ADHD group. The dif-
ferences between all groups were significant ( p <.01).
Next, to determine the proportion of children that
the CSBQ could correctly classify, a discriminant
analysis was performed. The five groups were com-
pared on the five scales and four significant discrimi-
nant functions could be determined. A study of the
structure matrix of the five scales, the four functions,
and their correlations showed that the first discriminant
function ( p <.001) demonstrated high correlations with
all scales (.52–.85), which can therefore be interpreted
as a general psychopathology function. The second dis-
criminant function ( p <.001) showed a strong (nega-
tive) relationship (.60) with Acting-out behaviors
(Scale 1) and a positive relationship (.34) with Social
Contact Problems. This function seems to describe
withdrawn behaviors. The third discriminant function
Children’s Social Behavior Questionnaire
( p <.001) had a negative correlation (−.44) with Social
Insight Problems and a positive correlation (.41) with
the Anxious/Rigid scale. In this case, children with high
scores on the Social Insight scale received, in general,
low scores on the Anxious/Rigid scale and vice versa.
Finally, the fourth discriminant function ( p <.05) had
the strongest relationship with Stereotypical behaviors
(.75) and Axious/Rigid behaviors (.49). The problems
described in both scales seemed to occur together.
Table VII showes the classification results on the
basis of the four discriminant functions. Nearly 50 %
of the original five groups could be correctly classified.
The results show that it was difficult to predict group
membership of especially the heterogeneous Clinical
Control group and the hetreogeneous PDDNOS group
on the basis of the four discriminant functions.
DISCUSSION
The main aims of this study were to examine the
psychometric characteristics of the CSBQ using a large
sample of children with a variety of child-psychiatric di-
agnoses and to compare the results for this instrument
with those obtained by others applied to the same groups.
A high-functioning autistic, a PDDNOS, an ADHD
group, and a clinical control group were included in this
study. The results obtained are encouraging with respect
to both reliability and validity.
On the basis of a principal component analysis, the
CSBQ items were grouped into five factors. On the basis
of these factors, five homogeneous scales were con-
structed that refer to items describing Acting-out Prob-
lems, Social Contact Problems, “Social Insight Problem,
Anxious/Rigid behaviors, and Stereotypical behaviors.
These five scales seemed to be relatively independent of
each other. They showed high internal reliability (with
coefficient α ranging from .76 to .92), satisfactory
test–retest reliability and satisfactory interrater reliabil-
ity. On the basis of the results, it can be concluded that
Table VII. Classification Results of the Discriminant Functions
Predicted group membership (%)
PDDNOS ADHD
PDDNOS 27.6 23.7
ADHD 8.8 64.9
Autism 14.7 4.2
Clinical controls 10.9 21.0
Normal controls 0.9 5.2
327
it is possible to design interpretable and relatively inde-
pendent scales by means of the CSBQ items.
A comparison with other instruments demonstrated
that the central problems described in the five scales cor-
responded with similar scales in the other instruments.
However, looking at the items of the scales of the CSBQ
in comparison with the items of the ABC (as autism in-
strument) and the CBCL (as checklist for general psy-
chopathology), the CSBQ items seem to refer more to
milder PDD features and to the detailed descriptions of
behaviors provided for children with milder PDDs.
Comparisons of the scales of the CSBQ with scales
of the DSM-IV criterion questionnaire for clinicians re-
sulted in weaker correlations. An explanation for this may
be that the DSM-IV criterion questionnaire contains items
that describe the core problems of Autism. The CSBQ,
on the other hand, embraces not only (milder) PDD prob-
lems but also PDD-related problems, such as motor prob-
lems and anxiety problems. Another possible explanation
for the weaker correlations is that the DSM-IV criterion
questionnaire is a different measurement method. It is
scored by a clinician, instead of parents/caregivers, and
this might influence the scores. Examination of the total
scores and the subscale scores of the different groups on
the DSM-IV criterion questionnaire suggests that the
differences between the problems of the PDDNOS group
and the Autism group are merely quantitative.
Comparison of the scores of the high-functioning
autism group, the PDDNOS group, and the ADHD group
on the five scales of the CSBQ indicate that the scales
describing Social Contact Problem, Anxious/Rigid, and
Stereotypical behaviors refer most specifically to PDD
problems. In each case, the Autism group received the
highest scores, followed by the PDDNOS group. These
sets of scores differed significantly from those of the
ADHD group.
The scale describing Acting-out behavior applied
most strongly to the ADHD group. The items of this
scale seem to refer to especially hyperactive and dis-
ruptive behaviors. Since acting-out behaviors are not
Autism Clinical controls Normal controls
28.9 10.5 9.2
6.4 9.4 10.5
68.4 7.4 5.3
13.3 23.3 31.6
0.9 8.2 84.9
328
normally included in descriptions of PDD problems,
the scores of both the PDDNOS group and the Autism
group on this scale are surprisingly high. These prob-
lem behaviors, while occurring in all three groups, may
arise through different underlying processes. For ex-
ample, the appearance of disobedient behaviors in chil-
dren with PDD problems may be caused by different
underlying processes from those in children with
ADHD (e.g., disobedience because of not understand-
ing social cues vs. inattentiveness and impulsiveness).
The comparison of the PDDNOS group and the
ADHD group on the Social Insight Problems scale re-
vealed that the PDDNOS group scored higher, though
not significantly so. The items on the scale refer to an
inability to pick up social cues, which leads to diffi-
culties in interpersonal relationships. They particularly
concern the social information processing and under-
standing of social cues, which are seen as PDD-spe-
cific problems. The significantly higher scores obtained
by the Autistic children are in agreement with this in-
terpretation. A striking result, however, is the relatively
high score of the ADHD group. Several authors have
speculated on social problems in children with ADHD.
Barkley (1997), for instance, concluded on the basis of
several studies that there is less reciprocity in the so-
cial exchanges of children with attentional problems.
Greene et al. (1996) claimed that “socially disabled”
children with ADHD may represent a meaningful sub-
type with varying course and prognosis. Cantwell
(1996) also described a type of comorbidity in children,
adolescents, and adults with the ADD syndrome as
“lack of social savoir-faire.” Therefore, a possible ex-
planation for this finding might be that, at least for a
part of the ADHD group, social insight problems seem
to be characteristic. Another explanation might be that
it is particularly the attention-related items of this scale
that cause the relatively high scores of the ADHD
group. Again, one might assume that, although the same
problem behaviors are described in the three clinical
groups, different causes may effect these deficits.
On the basis of the comparisons of the different
groups on the scale scores of the CSBQ it can be con-
cluded that the PDDNOS group cannot simply be re-
garded as a milder form of autism. If this assumption
were to be correct, the Autism group would receive
higher scores on all scales in comparison with the
PDDNOS group. The differences would also be ap-
proximately the same for each scale. This is not the
case, in fact the PDDNOS group received even higher
scores than the Autism group on the Acting-out scale.
Besides, the discriminant analysis on the five scales in
all groups showed that on the basis of four significant
Luteijn, Jackson, Volkmar, and Minderaa
discriminant functions the children of the PDDNOS
group were predicted to belong to approximately the
same extent in the Autism group, the PDDNOS group,
and the ADHD group. This might suggest the existence
of subgroups within the category, with relations to
autism and ADHD in particular. If this interpretation
of the results is correct, it could also have consequences
for treatment. Further research is necessary in order to
develop more insight into the differences between the
groups and between children within the same category
but with different scores.
This study indicates that the CSBQ offers impor-
tant possibilities for research and for clinical practice.
There are, however, further aspects of the questionnaire
that require investigation. For example, the relation-
ship of variables such as age, gender, or IQ with the
scores on the CSBQ is not yet known. The differences
between the groups in this study may possibly be ex-
plained by the effects of one or more of these variables
on the scores. It is also necessary to obtain scores for
mentally retarded children. In future studies, it would
be interesting to examine and compare the scores of
these children with other clinical groups.
Another suggestion for further research is to ex-
amine the reliability, validity, and the construction of
the scales separately for a PDD group, since that is the
population that the instrument was designed to iden-
tify. However, this would require the use of very large
samples, which is practically hard to fulfill. Besides, a
disadvantage of the inclusion of large samples in a
study is that it is much more difficult to use standard-
ized interviews or to determine full-scale IQs for the
total sample.
A further consideration is that the groups were
formed on the basis of clinical diagnosis only. At pre-
sent, it remains impossible to classify children with
milder forms of autism on the basis of standardized in-
struments. In this study a DSM-IV criterion list was
used to support the classifications. Future research on
the PDDNOS category requires descriptions of the
problems of this group which are based on standard-
ized instruments. As Rutter (1996, 1999) has stated, the
relationship of autistic-like conditions (such as
PDDNOS) to strictly defined autism requires specifi-
cation and may have particular importance for genetic
studies. According to Volkmar et al. (1997), behavioral
research depends on well-defined groups of patients
and rigorous application of diagnostic methodologies
as well.
The development of instruments such as the CSBQ
can be seen as a step in this direction. The CSBQ in
combination with cluster analytic techniques could also
Children’s Social Behavior Questionnaire
be useful to examine whether there are subgroups
within the autism spectrum. Further research is neces-
sary to deal with such issues and also to provide de-
tailed information concerning the possible contribution
of the CSBQ to diagnosis, classification, and treatment.
The present research indicates that it has considerable
promise for application to each of these fields.
REFERENCES
Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/
4-18 and 1991 profile. Burlington: University of Vermont
Department of Psychiatry.
American Psychiatric Association. (1994). Diagnostic and statistical
manual of mental disorders (4th ed.). Washington DC: Author.
Barkley, R. A. (1997). Attention deficit hyperactivity disorder: A
handbook for diagnosis and treatment (2nd ed.). New York:
Guilford.
Bartko, J. J. (1976). On various intraclass correlation reliability co-
efficients. Psychology Bulletin, 83, 762–765.
Bolton, P., MacDonald, H., Pickles, A., Rios, P., Goode, S., Crow-
son, M., Baily, A., & Rutter, M. (1994). A case-control family
history study of autism. Journal of Child Psychology and Psy-
chiatry, 35, 977–900.
Buitelaar, J. K., & Van der Gaag, R. J. (1998). Diagnostic rules for chil-
dren with PDD-NOS and Multiple Complex Developmental Dis-
order. Journal of Child Psychology and Psychiatry, 39, 911–919.
Cantwell, D. P. (1996). Attention Deficit Disorder: A review of the
past 10 years. Journal of the American Academy of Child and
Adolescent Psychiatry, 35, 978–987.
Castelloe, P., & Dawson, G. (1993). Subclassification of children
with autism and pervasive developmental disorder: A question-
naire based on Wing’s subgrouping scheme. Journal of Autism
and Developmental Disorders, 23, 229–242.
Cohen, D., Paul, R., & Volkmar, F. (1987). Issues in the classifica-
tion of pervasive developmental disorders and associated con-
ditions. In: D. J. Cohen & A. M. Donnellan (Eds.), Handbook
of autism and pervasive developmental Disorders (pp. 20 – 40).
New York: Wiley.
Cohen, D. J., Towbin, K. E., Mayes, L., & Volkmar, F. R. (1994).
Developmental psychopathology of multiplex developmental
disorder. In S. L. Friedman & H. C. Haywood (Eds.), Devel-
opmental follow-up: Concepts, genres, domains and methods
(pp. 155–179) New York: Academic Press.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure
of tests. Psychometrika 16, 671–684.
Greene, W. R., Biederman, J., Faraone, S., Ouellette, B. A., & Grif-
fin, S. M. (1996). Toward a new psychometric definition of so-
cial disability in children with attention-deficit hyperactivity dis-
order. Journal of the American Academy of Child and Adolescent
Psychiatry, 35, 571–578.
Jensen, V. K., Larrieu, J. A., & Mack, K. K. (1997). Differential di-
agnosis between attention-deficit/hyperactivity disorder and per-
vasive developmental disorder-not otherwise specified. Clinical
Pediatrics, 36, 555–561.
Kiers, H. A. L. (1990). SCA, A program for simultaneous compo-
nents analysis of variables measured in two or more popula-
tions. University of Groningen: IEC Programma.
Krug, D. A., Arick, J. R., & Almond, P. J. (1980). Autism screening
instrument for educational planning. Portland, OR.: ASIEP.
Le Couteur, A., Rutter, M., Lord, C., Rios, P., Robertson, S., Hold-
grafer, M., & McLennan, J. D. (1989). Autism Diagnostic In-
terview: A semistructured interview for parents and caregivers
of autistic persons. Journal of Autism and Developmental Dis-
orders, 19, 363–387.
329
Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Maw-
hood, L., & Schopler, E. (1989). Autism Diagnostic Observation
Schedule. A standardized observation of communicative and so-
cial behavior. Journal of Autism and Developmental Disorders,
19, 185–212.
Luteijn, E. F., Jackson, A. E., Volkmar, F. R., & Minderaa, R. B.
(1998). The development of the Children’s Social Behavior
Questionnaire: Preliminary data. Journal of Autism and Devel-
opmental Disorders, 28, 559–565.
Mayes, L. C., Volkmar, F. R., Hooks, M., & Cicchetti, D. (1993).
Differentiating pervasive developmental disorder not otherwise
specified from autism and language disorders. Journal of Autism
and Developmental Disorders, 23, 79–90.
Prior, M., Leekman, S., Ong, B., Eisenmajer, R., Wing, L., Gould,
J., & Dowe, D. (1998). Are there subgroups within the autistic
spectrum? A cluster analysis of a group of children with autis-
tic spectrum disorders. Journal of Child Psychology and Psy-
chiatry, 39, 893–902.
Rutter, M. (1999). The Emanuel Miller Memorial Lecture 1998.
Autism: Two-way interplay between research and clinical work.
Journal of Child Psychology and Psychiatry, 40, 169–188.
Rutter, M. (1996). Autism research: Prospects and priorities. Jour-
nal of Autism and Developmental Disorders, 26, 257–276.
Rutter, M., & Schopler, E. (1992). Classification of pervasive de-
velopmental disorders: some concepts and practical considera-
tions. Journal of Autism and Developmental Disorders, 22,
459–482.
Schopler, E., Reichler, R. J., De Vellis, R. F., & Daly, K. (1980). To-
ward objective classification of childhood autism: Childhood
Autism Rating Scale (CARS). Journal of Autism and develop-
mental Disorders, 10, 91–103.
Schopler, E., Reichler, R. J., & Renner, B. R. (1988). The Childhood
Autism Rating Scale (CARS). Los Angeles: Western Psycho-
logical Services.
Szatmari, P. (1992). The validity of autistic spectrum disorders: A
literature review. Journal of Autism and Developmental Dis-
orders, 22, 583–600.
Towbin, K. E. (1997). Pervasive developmental disorders not other-
wise specified. In D. J. Cohen & F. R. Volkmar (Eds.), Hand-
book of autism and pervasive developmental disorders. New
York: Wiley.
Towbin, K. E., Dykens, E. M., Pearson, G. S., & Cohen, D. J. (1993).
Conceptualizing borderline syndrome of childhood schizophre-
nia as a developmental disorder. Journal of the American Acad-
emy of Child and Adolescent Psychiatry, 32, 775–782.
Van der Gaag, R. J. (1997). De aan autisme verwante contact sto-
ornis. In M. A. H. Mulders, M. A. T. Hansen, & C. J. A. Roosen
(Eds.), Autisme: Aanpassen en veranderen (pp. 15–23). Assen/
Maastricht: van Gorcum.
Van der Gaag, R. J., & Van den Ban, E. F. (1993). Psychofarmaca
bij kleuters en kinderen; een landelijke inventarisatie. In F. Ver-
hey (Ed.), Klinische Kinder- en Jeugdpsychiatrie in Nederland
(pp. 201–208). Assen/Maastricht: van Gorcum.
Van der Gaag, R. J., Buitelaar, J. K., Van den Ban, E., Bezemer, M.,
Njio, L., & Van Engeland, H. (1995). A controlled multivariate
chart review of multiple complex developmental disorder. Jour-
nal of the American Academy of Child and Adolescent Psychi-
atry, 34, 1096–1106.
Verhulst, F. C., Berden, G. F. M. G., & Sanders-Woudstra, J. A. R.
(1985). Mental health in Dutch children. II. The prevalence of
psychiatric disorder and relationship between measures. Acta
Psychiatrica Scandinavica, 72 (Suppl.342).
Verhulst, F. C., Van der Ende, J., & Koot, J. M. (1996). Handleid-
ing voor de CBCL/4-18 (Nederlandse versie). Rotterdam:
Afdeling kinder- en jeugdpsychiatrie, Sophia kinderziekenhuis/
Academisch Ziekenhuis Rotterdam/Erasmus Universiteit
Rotterdam.
Volkmar, F. R., Klin, A., & Cohen, D. J. (1997). Diagnosis and clas-
sification of autism and related conditions. In D. J. Cohen &
330
F. R. Volkmar (Eds.), Handbook of autism and pervasive
developmental disorders. New York: Wiley.
Volkmar, F. R., Cicchetti, D. V., Dykens, E., Sparrow, S. S., Leck-
man, J. F., & Cohen, D. J. (1988). An evaluation of the Autism
Behavior Checklist. Journal of Autism and Developmental Dis-
orders, 18, 81–97.
Volkmar, F. R., Klin, A., Siegel, B., Szatmari, P., Lord, C., Camp-
bell, M., Freeman, B. J., Cicchetti, D. V., Rutter, M., Kline, W.,
Buitelaar, J., Hattab, Y., Fombonne, E., Fuentes, J., Werry, J.,
Stone, W., Kerbeshian, J., Hoshino, Y., Bregman, J., Loveland,
K., Szymanski, L., & Towbin, K. (1994). Field trial for Autis-
tic Disorder in DSM-IV. American Journal of Psychiatry, 151,
1361–1367.
Luteijn, Jackson, Volkmar, and Minderaa
Wing, L. (1988). The continuum of autistic characteristics. In E.
Schopler & G. Mesibov (Eds.), Diagnosis and assessment in
autism (pp. 91–110). New York: Plenum Press.
Wing, L. (1997). Syndromes of Autism and Atypical Development.
In D. J. Cohen & F. R. Volkmar (Eds.), Handbook of autism and
pervasive developmental disorders. New York: Wiley.
Wing, L., & Gould, J. (1979). Severe impairments of social interac-
tion and associated abnormalities in children: Epidemiology and
classification. Journal of Autism and Developmental Disorders,
9, 11–29.
World Health Organization. (1993). The ICD-10 classification of
mental and behavioural disorders. Diagnostic criteria for re-
search. Geneva: WHO.