Eur Child Adolesc Psychiatry (2007)

16:484–494 DOI 10.1007/s00787-007-0624-1 ORIGINAL CONTRIBUTION

Khrista Boylan Comorbidity of internalizing disorders

Tracy Vaillancourt
Michael Boyle in children with oppositional defiant
Peter Szatmari
disorder

j Abstract Oppositional defiant comorbidity with ODD is present

Accepted: 29 March 2007
Published online: 24 September 2007
K. Boylan (&) Æ M. Boyle Æ P. Szatmari
Department of Psychiatry and Behavioural
Neurosciences
McMaster University
1200 Main St. West
Hamilton (ON), Canada L8N 3Z5
Tel.: +1-905/521-2100
Fax: +1-905/574-6665
E-Mail: boylank@mcmaster.ca
T. Vaillancourt
Department of Psychology
McMaster University
1280 Main St. West
Hamilton (ON), Canada L8S 4K1
disorder (ODD) is often comorbid
with other psychiatric disorders in
childhood. Its association with
attention deficit hyperactivity dis-
order and conduct disorder has
been well studied. Recent studies
suggest that children with ODD
have substantial comorbidity with
anxiety and depressive (internal-
izing) disorders, as well. Identify-
ing the pattern of internalizing
comorbidity with ODD in child-
hood and adolescence and how
this varies across age and gender
may help to identify mechanisms
of such comorbidity. This sys-
tematic review presents evidence
on the association of internalizing
disorders with ODD across child-
hood and adolescence. Data from
cross-sectional and longitudinal
studies in clinic, community and
epidemiologic samples are con-
sidered separately. Findings sug-
gest that while internalizing
at all ages, the degree of comor-
bidity may vary over time in
particular groups of children. Girls
and boys appear to have different
patterns of ODD comorbidity with
either anxiety or depression, as
well as ages of onset of ODD,
however more large studies are
required. Children with ODD in
early life require further study as
they may be a subgroup at in-
creased risk for anxiety and affec-
tive disorders. This could have
important implications for the
treatment of these ODD children
and the prevention of sequential
comorbidity.
j Key words oppositional defiant
disorder – internalizing disorder –
childhood – prevalence – epide-
miology – comorbidity

ODD is considered a disruptive behaviour disor-

Introduction
Oppositional defiant disorder (ODD) is one of the
most common psychiatric disorders of childhood.
The Diagnostic and Statistical Manual of Mental
Disorders [1] defines ODD as a pattern of negativistic,
hostile and deviant behaviour that is severe enough to
impair the child’s functioning for at least 6 months
ECAP 624
der, because many children with ODD have similar
cognitive and social deficits and behaviour problems
as children with other behaviour disorders such as
attention deficit hyperactivity disorder (ADHD) and
conduct disorder (CD) [24, 47]. Although many
studies suggest that ODD is associated prospectively
with ADHD and CD [8, 33], more than 60% of chil-

and does not occur solely during an episode of psy- dren with ODD do not have ADHD in childhood [4]
chotic or mood disorder. and approximately only 10% will develop conduct
 K. Boylan et al.
ODD and internalizing disorders
disorder [34]. ODD is also a common comorbidity in
studies of mood and anxiety disorders suggesting that
there is also important links with internalizing
pathology. These possible links are not surprising,
given that children with ODD are easily annoyed, have
problems with affect regulation and are often de-
scribed as moody and irritable. The degree of com-
orbidity and the mechanism for comorbidity between
ODD and internalizing disorders are unclear.
The primary focus of this paper is to describe the
observed relationship between ODD and the inter-
nalizing disorders of anxiety (all types) and major
depression. We argue that there are important sub-
stantive reasons that ODD might co-occur with
internalizing disorders that are independent of ODD’s
association with CD or ADHD. There are at least four
key mechanisms for such co-association: (a) ODD is a
prodrome for evolving internalizing disorders, (b)
ODD and internalizing disorders arise from common
risk factors, (c) ODD is a risk factor for an internal-
izing disorder, (d) ODD and internalizing comorbid-
ity may co-occur because of measurement errors or
reporting biases. As a way of examining these possible
mechanisms and associations, a systematic literature
review is undertaken in which the following questions
are addressed:
(1) What is the prevalence of ODD in clinical and
epidemiologic samples?
(2) What is the degree of comorbidity between ODD
and internalizing disorders?
(3) Does comorbidity between ODD and type of
internalizing disorder vary between clinical and
epidemiologic samples, and between cross-sec-
tional and longitudinal samples?
(4) Does age or gender of the child influence the
pattern of this comorbidity?
Search strategy
Four literature databases, PubMed, CINAHL, Psychi-
nfo, Medline, were searched for English language
studies of children under the age of 18. The first
search was for articles that reported the epidemiologic
or clinical prevalence of ODD. Search terms included
‘‘oppositional defiant disorder’’ or ‘‘ODD’’, combined
with each of the search terms ‘‘prevalence’’ and ‘‘ep-
idemiologic’’/‘‘population’’/‘‘community’’ or ‘‘clinic’’
study as appropriate. Studies were included if they
used the DSM-III, DSM-IIIR, DSM-IV or DSM-IV-TR
diagnostic criteria. The number of articles retrieved
from this search were 509 for epidemiologic/popula-
tion/community-based studies and 141 for clinical
485
studies. The second search was done to identify epi-
demiologic and clinical studies that reported on the
prevalence of various internalizing disorders in chil-
dren who had ODD, with or without other comorbid
disorders. Search terms included all of the above
terms, as well as ‘‘behaviour disorder’’ in the place of
oppositional defiant disorder, as well as including
various combinations of ‘‘major depressive disorder’’,
‘‘dysthymia’’, ‘‘anxiety disorder’’, ‘‘internalizing dis-
order’’. We included the search term ‘‘behaviour
disorder’’ as many studies of internalizing disorder in
children with ADHD exist, and only a subset of these
also report on comorbidity with ODD. The number of
articles retrieved in this search was 147 community
and 69 clinical studies. The article abstracts were read
to determine if data on ODD and internalizing com-
orbidity were reported, and the articles that addressed
any of the four questions were reviewed. The refer-
ence lists from these articles were then used to locate
other studies. About 28 articles resulted from the
search and were reviewed in detail (Tables 1–3).
Paper organization
Studies reporting prevalence of ODD by age and
gender are first presented, followed by studies
reporting comorbidity between ODD and internaliz-
ing disorder. These latter studies are presented along
two study designs: cross-sectional studies (reporting
concurrent comorbidity) and longitudinal studies
(reporting sequential comorbidity). Within each sec-
tion, epidemiologic studies are presented prior to
clinical studies as the former provide an estimate of
comorbidity that is less likely to be influenced by
referral bias or severity of illness [5]. When a study
reports both cross-sectional and longitudinal data, the
study’s results are discussed as longitudinal data so
that the study results are only reported once. Preva-
lence data obtained from all studies reviewed are
presented in tables.
Several points about the measurement of comor-
bidity must be kept in mind when reviewing the re-
sults. Measurement of comorbidity relies of the
accuracy of the diagnostic process since comorbidity
implies that two distinct disorders are present. Each
disorder should meet symptom criteria of the DSM or
ICD, but also must reflect the presence of impairment
related to the diagnosis in question. Thus, if diagnoses
are made using a parent checklist without regard to
the impairment, then estimates of comorbidity are
likely to be higher. It should be noted that all but two
[10, 18] of the epidemiologic studies included in this
review employed a semi-structured diagnostic inter-
view using trained interviewers and one or more
486 European Child & Adolescent Psychiatry (2007) Vol. 16, No. 8
Ó Steinkopff Verlag 2007

Table 1 Cross-sectional community and epidemiologic studies of odd and internalizing comorbidity

Author Mean Prevalence of ODD Prevalence of OR (depression Prevalence of OR (anxiety disorder

age depression in in ODD) anxiety disorder in ODD) with
(N) children with in children 95%CI
ODD with ODD

Anderson et al. [2] 11 (792) 5.7 15.3 38.3 (10.4–41)a 26.4 5.4 (2.9–9.9)a
Angold et al. [4] 9,11,13 (1,071) 2.8 (DSM 3R)
4.9 (DSM 4)
Bird et al. [9] 9–16 (222) 9.9 45.4 18.4 (6.1–55.3)a 55.3
Breton et al. [10] 2,400
Age 6–8 2.6
Age 9–11 2.6
Age 12–14 3.0
Age 6–14 2.7
Carlson et al. [14] 2,984 (6–9) 6.8 1.0 (f)
1.0 (m)
Cohen et al. [16] 975
Age 10–13 14.2 (m) 10.4 (f)
Age 14–16 15.4 (m) 15.6 (f)
Age 17–21 12.2 (m) 12.5 (f)
Costello et al. [17] 9–13 (1,420) 3.1 (m) 16.7 (5.9–47.8) 0.8 (0.5–2.5)
2.1 (f) 20.7 (8.8–48.8) 3.3 (1.7–6.5)
Ersan et al. [18] 6–15 (1,425) 11.0
Fleitlich-Bilyk [19] 7–14 (1,251) 3.2 4.6 (1.9–7.6) 3.8 (1.3–6.1)
Ford et al. [20] 5–15 (10,438) 2.3 (3.2 (m)/1.4 (f)) 1.8 (0.8–4.4) 6.4 (3.7–8.1)
Gadow and Nolan [21] 4 (595) Parent: 9.1 1.4 1.0
Teacher: 12.1
Kashani et al. [27] 14–16 (150) 6.0 23.5 52.5 (10.3–268)a 40.9 21.5 (5.8–79)a
Simonoff et al. [45] 8–16 (2,762 twins) 3.4 (3.9 m, 3.0 f) 11.2 (4.6–25.6) 3.0 (1.6–3.4)
Lahey [32] 9–17 (1275) 4 (m) 2 (f)
Maughan et al. [37] 5–15 (10,438) 3.2 (m) 2.4 (m) 4.0 (1.2–13.2) 14.2 (m) 4.8 (2.8–8.4)
1.4 (f) 4.4 (m) 2.1 (0.5–8.9) 22.2 (f) 7.8 (4.1–14.6)
McGee [38] 14–18 (943) 1.7 15.3 5.6 (2.7–11.2)a 7.1 0.6 (0.3–1.4)a
Rowe et al. [40] 9–16 (1420) 1.8 14.6 18.4 (7.7–44.1) 7.3 3.7 (1.8–7.5)
a
OR estimates were published in Angold et al. [5]

informants, attesting to the validity of the term co-
morbid in the clinical sense. In the clinical studies,
several did not require that the presence of impair-
ment be present for the comorbid diagnosis to be
made [7, 22, 25, 43], thus these results need to be
interpreted as potentially providing an overestimate
of comorbidity.
A second point to bear in mind is the statistical
presentation of comorbidity. Comorbidity can be de-
fined between categories, or between disease states that
may be measured as continuous variables [5]. The de-
gree of comorbidity between two categorically-defined
disorders will be reported as the odds ratio (OR). For
comorbidity to be statistically significant, the co-
occurrence of two disorders in a given sample must be
greater than chance alone would predict, hence the 95%
confidence interval (CI) surrounding the OR estimate
would not contain a value of 1. OR calculations can also
be adjusted (‘‘adjusted OR’’) for the presence of other
confounding conditions. This provides an estimate of
the increased likelihood that the particular comorbid-
ity occurs above chance alone, taking into account of
those confounding conditions. It should be noted that
in the tables, some studies do not have an associated OR
or there is no prevalence data for each comorbid con-
dition. These gaps occur because the data were not
provided in the text of these papers.
Only three [14, 22, 32] studies in this review report
oppositional or internalizing symptom severity using
a categorical independent variable (ODD) and a
continuous dependent variable (internalizing disor-
der) scale. In these papers, the degree of comorbidity
is calculated using an estimate of effect size (ES) [15],
which allows the comparison of how much the pres-
ence of oppositionality effects or increases the average
internalizing score in a group of children. The for-
mulae for OR and ES are presented in the Appendix 1.
Two disorders can occur at the same time or age,
whereby comorbidity is referred to as concurrent.
Comorbidity can also occur over the lifetime of the
individual whereby it can be termed lifetime, sequential
or longitudinal . When the disorders that are comorbid
belong to the same diagnostic grouping (i.e. external-
izing disorders), the comorbidity is homotypic and if
 K. Boylan et al. 487
ODD and internalizing disorders

Table 2 Cross-sectional clinical studies of odd and internalizing comorbidity

Author Age (N) % ODD Prevalence (%) of OR (depression Prevalence (%) of OR (anxiety in
depression in ODD in ODD) with 95%CI anxiety disorder in ODD ODD) with 95%CI

Gadow and Nolan [21] 3–6 (224) Parent: 37.0 * 0.5 * 0.7
Teacher: 26.5
Garland and Garland [22] 6–12 (145) 45.0 50.0 50.0
Greene [23] 11 (1600) 40.0 30.0 3.2 (2.6–4.7) 38.0 1.5 (1.2–1.9)
Keenan [28] 2–5 (79) 59.5
Kuhne and Schachar [30] 8 (91) 50.4 * 0.6 (f) *
0.6 (m)
Rey [43] 12–16 (2093) 28.0 24.0 1.4 (1.1–1.8)
Avg. OR (unweighted) 6.0 (three studies) 8.9 (two studies)

*Disorder prevalence estimate could not be calculated from the data

Data in italics represents an effect size estimate

Table 3 Longitudinal community, clinical and epidemiologic studies of odd and internalizing comorbidity

Study Age %ODD Prevalence depression RR (95%CI) depression Prevalence anxiety RR (95%CI) anxiety

August et al. [6]a 6–10 (168) Year 1: 31.5

4 year study Year 4: 32.7
a
Beiderman et al. [7] 6–17 (128) 65.0 57.0 8.8 68.0 4.0
4 year study
Kim-Cohen [29]b 11 (1,037) 1.5 (1.0–2.2) 2.9 (2.1–4.1)
15 year study
Lavigne [34]a 2–5 (280) Year 1: 100.0 0 0
5 year study Year 2: 39.5 4.6 7.4
Year 5: 24.0 4.0 24.3
Speltz [45]a 4–5 (92) Year 1: 100.0 0 7.6
2 year study Year 3: 76.0 4.0 10.1

Community study, a Clinical study, b

Epidemiologic study

from different groupings (i.e. internalizing and exter- Clinical samples

nalizing disorder) it is heterotypic [5]. Both types of
comorbidity are presented in this article.
j Prevalence studies of ODD
Community samples
About 18 community studies were identified
(Table 1). These studies report prevalence rates of
2.6–15.6% for ODD across childhood and adoles-
cence, with 13 of these studies reporting prevalence
rates between 3% and 6%. Preschool prevalence was
higher ranging from 9% to 12% [32].
Gender differences have been inconsistent across
smaller studies [10, 17, 34]. In the largest published
study (N = 10,000) [39], the prevalence of ODD ap-
pears greater in boys (3.2%) than girls (1.8 %) across
ages 6–16 [39]. This sex difference appears to vary by
age, with higher rates in boys prior to adolescence, and
no sex differences in adolescence. There may also be
peaks in prevalence around age 7 for boys [39], and a
second peak for both sexes at age 14–15 [18, 40].
In Table 2, we present the six studies (two including
preschool aged children) that reported on the preva-
lence of ODD in clinical samples.
The prevalence of ODD ranged from 28% to 65%.
The highest rates are reported in samples of children
from behaviour problem or ADHD clinics [25, 28, 30].
Rates of ODD or comorbid ODD in clinic samples are
typically higher than in the community because the
diagnosis of ODD may lead to clinic attendance in
hopes of dealing with associated difficulties in child
behavioural management.
j Prevalence studies of internalizing comorbidity
in children with ODD
Concurrent comorbidity in community samples
Two studies of community preschoolers show signif-
icantly higher mean scores of parent-rated [22] and
teacher-rated [14] major depression and separation
anxiety symptoms in preschoolers with ODD than in
those without ODD.
488 European Child & Adolescent Psychiatry (2007) Vol. 16, No. 8
Ó Steinkopff Verlag 2007
A subset of the epidemiologic studies cited pre-
viously also support the finding of a higher preva-
lence of internalizing comorbidity in older children
and adolescents with ODD. In these studies, the
prevalence of comorbid major depression and ODD
ranged from 15% to 46% which results in the
average OR of major depression with ODD of 17.5
(range: 4–52, with overlapping 95% CIs for all
estimates), while prevalence of comorbid anxiety
disorder of any type and ODD ranged from 7% to
14% (average OR = 3.0; range: 0.6–22; 2, 9, 29, 40,
45). As noted by Angold et al. [5] all but two [2,
29] of these studies grouped diagnoses of ODD and
CD to calculate the prevalence of ODD which makes
it difficult to obtain accurate comorbidity estimates
for ODD.
Three other epidemiologic studies [17, 21, 49] re-
ported increased adjusted odds of association be-
tween either major depression [17, 49] or any anxiety
disorder [21], but not both, in the same study with
ODD.
A gender-specific analysis of comorbidity [39]
found a significant association between either anxiety
disorder or major depression with ODD in boys, and
with any anxiety disorder and ODD only in girls.
Costello et al. [17] reported that the adjusted odds of
association of ODD with major depression signifi-
cantly increased for girls and boys but the adjusted
OR for ODD and any anxiety disorder was not sig-
nificant for either sex.
Concurrent comorbidity in clinical samples
There are six cross-sectional clinic studies reporting
on comorbidity, including one study of preschool-
ers. Using a DSM-IV based symptom checklist,
Gadow and Nolan [22] found a significantly higher
mean of parent-rated major depression or separa-
tion anxiety symptoms in preschoolers with, than
without, ODD (Effect size = 0.5 and 0.7, respec-
tively). These estimates of effect size [15] suggest
that the mean internalizing symptom score of a
child with ODD lies at the 75th percentile of the
distribution of internalizing symptoms scores of
children without ODD; a large effect of ODD on
internalizing symptom report.
Similar findings have been reported in studies of
older children. The two large clinics [25, 43] docu-
ment an increased odds of major depression [25, 43]
as well as a composite prevalence of multiple anxiety
disorders [25] in children with ODD compared to
controls without any disorder. A significant increased
effect of the presence of ODD on internalizing
symptom report was found when using teacher rating
scale reports on presence of internalizing symptoms
in 7–9 year olds [30].
Summary
These cross sectional studies suggest that ODD is
associated with major depression and anxiety dis-
orders based on findings of significant adjusted ORs
and moderate effect sizes. Not all studies have found
both classes of internalizing disorders to be associ-
ated with ODD, so characteristics of the study’s
sample such as age or gender may influence these
estimates. For example, ODD and major depression
comorbidity may be greater in a sample of adoles-
cents, for whom the prevalence of major depression
is greater. Not all clinical studies used DSM diag-
noses or required the presence of impairment to be
present for diagnosis, potentially increasing rates of
comorbidity from these studies [25, 43]. No defini-
tive conclusions can be made about such age and
gender influences on comorbidity as the magnitude
of ORs varies between these cross-sectional studies.
j Longitudinal studies of internalizing comorbidity
in children with ODD
Community samples
One study describes the longitudinal course of inter-
nalizing symptoms in community children with ODD.
August et al. [6] recruited 308 children from schools
who were aged 7–11 and scored greater than 1.75
standard deviations on the Revised Conner’s Hyper-
activity Index. They also followed a random sample of
low scorers as a comparison group. Children were
classified using their ODD status at intake and 4 years
later as ‘‘persisters’’ (consistently elevated scores),
‘‘desisters’’ (scores reducing over time), ‘‘late onset-
ters’’ and ‘‘resisters’’ (low scores throughout). Mean
symptom counts were also determined at each time
point for other comorbid disorders.
About 18% had persistent ODD, 15% were
‘‘desisters’’ 51% were ‘‘resisters’’, and 16% developed
ODD 4 years later (late onsetters). Two striking
patterns were evident. Both the persistent and late
onset ODD groups had significantly greater average
mood symptoms at 4 years than at the start of the
study. The persistent ODD group also had increased
CD and ADHD symptoms at the start, while new
onsetters did not. This suggests that although both
had increased depressive symptoms with age, the
late onset group were more likely to have mood and
ODD symptoms in the absence of prior psychopa-
thology. The children who never had ODD (resisters)
had the lowest rates of all comorbid symptoms at
both times and desisters were similar to this group
after 4 years, but at the start, had elevated CD and
anxiety symptoms.
 K. Boylan et al.
ODD and internalizing disorders
These preliminary results on a small sample dem-
onstrate that there is heterogeneity in the course of
ODD in children, which may indicate a different eti-
ology for each pattern of comorbidity. For example,
late onset ODD may signify the concomitant emer-
gence of a mood or anxiety disorder in many children,
whereas in children where ODD has been a long-
standing problem, it may reflect residual difficulties in
affect regulation relating to cognitive or attentional
problems.
Kim-Cohen [31] collected diagnostic data from a
birth cohort of 976 subjects. They were assessed at 26-
years of age and their current diagnosis was corre-
lated with their previous psychiatric diagnoses in
childhood. Prior ODD or CD (grouped as one diag-
nosis in this study) was significantly associated with
anxiety, depressive, substance abuse, schizophreni-
form and antisocial personality disorders in adult-
hood. Overall, the most common childhood diagnoses
were ODD/CD and any anxiety disorder, each in 20%
of these adults. Although this study found that ho-
motypic continuity from early anxiety to later anxiety
disorders was most common in this population, het-
erotypic continuity – particularly in the form of ODD
to internalizing disorder was also an important tra-
jectory. Adults with substance use disorder or
schizophreniform disorder also had higher rates of
early childhood ODD than adults with internalizing
disorders, suggesting that ODD might be a ‘‘gateway’’
disorder serving as an early indicator of risk of many
later mental illnesses.
Clinical samples
Four studies followed clinical populations of children
with ODD prospectively. One study [7] also included
children without ODD in their sample, allowing the
calculation of risk of developing internalizing disor-
der associated with ODD. In two cohort studies
including 232 ODD preschoolers, between 4% and
25% of children developed any mood and/or anxiety
disorder over 4 years of study [36, 50]. Of 260 chil-
dren with ADHD with or without ODD and CD, the
risk of new major depression and any anxiety disor-
der diagnosis by age 12 was significantly increased for
the group with ADHD and ODD compared to the
group with ADHD alone [7] (Risk Ratio = 7.1 for
mood and 2.8 for anxiety, respectively).
Burke et al. [11] described the clinical course of a
sample of 177 boys annually for 10 years. Using pre-
dictive regression models, they showed that ODD
preceded most other psychiatric disorders in their
sample, even when controlling for other covariates
and comorbid conditions. Of note, they found that
ODD was the only behaviour disorder at baseline that
predicted later internalizing disorder. Prevalence data
489
from this study is not displayed in Table 3 as it was
not cited in the paper.
Summary of clinic and community longitudinal
studies
These five longitudinal studies provide multiple in-
sights into course of comorbidity. Some 20% of ODD
children maintain a diagnosis of pure ODD over time.
Comorbidity can present early in life with 10–20% of
ODD children developing internalizing disorders as
preschoolers. Older youth with ODD appear to de-
velop internalizing comorbidity at higher rates than
younger ODD children and similarly aged children
without ODD. Whether the comorbidity is concurrent
or sequential was demonstrated in the preschool
studies which suggested that internalizing disorders
can present in children who previously had pure
ODD, but internalizing disorders are more common
in children who have persistent ODD.
Discussion
The papers reviewed in this article are few in number,
employed different sampling schemes and measure-
ment tools, and as such, conclusions must be con-
sidered preliminary. The discussion addresses
questions posed in the introduction followed by some
commentary on potential implications for clinical and
research practice.
j What is the prevalence of ODD in clinical and
epidemiologic samples?
ODD prevalence estimates were found to vary be-
tween 2% and 14% in epidemiologic samples and 28–
50% in clinic samples. As rates are consistent with
previous reviews of ODD [35], ODD appears to be the
most common psychiatric diagnosis in epidemiologic
samples, comparable in prevalence to ADHD mea-
sured as a combination of its three subtypes [5].
The phenomenon of higher rates of ODD in clinical
samples is well described in the literature and likely
occurs because of characteristics of clinical samples
including higher base rates of behaviour disorder and
functional impairment of the child and family upon
referral [3, 50]. Other influences on prevalence rates
include informant (parent, teacher or child) of
symptoms and methods employed to ascertain the
presence of disorder. For example, the higher esti-
mates reported in the Cohen [14] epidemiologic study
(range 10.5–15.6%) may be because either parent or
490 European Child & Adolescent Psychiatry (2007) Vol. 16, No. 8
Ó Steinkopff Verlag 2007
child symptoms were used for the diagnosis, and in
the Ersan et al. [18] study (prevalence 11.5%), the
diagnosis was obtained from a parent checklist
without clinician confirmation. As there may be little
diagnostic overlap between parent and teacher-rated
cases of ODD [22], studies not including both parent
and teacher to make a diagnosis may underestimate
prevalence of ODD [21].
j Does the comorbidity between ODD and type
of internalizing disorders vary between clinical
and epidemiologic samples?
In both epidemiologic and clinical samples, the sig-
nificantly elevated ORs between internalizing disor-
ders and ODD suggest that these comorbid conditions
co-occur more frequently than chance would predict.
The best estimate of comorbidity, presented here as
an unweighted average of the OR estimates of studies
providing such estimates, is 17.2 for major depression
with ODD and 5.4 for any anxiety disorder with ODD
in epidemiologic studies. In clinical studies the ORs
are more similar: 6.0 for major depression and 8.9 for
any anxiety disorder.
The reasons for the high rates of depression
comorbidity in epidemiologic samples are not entirely
clear, but several processes may be relevant. First, two
of the studies contributing the highest OR [2, 29] were
not adjusted for concurrent comorbidity, which
would inflate the estimate of average OR, however
clinical studies also did not control for concurrent
comorbidity. ORs in clinical samples are less likely to
be significant, or elevated, as such samples have
higher base rates for all types of disorders and com-
orbidity is also more likely to occur by chance alone.
Epidemiologic studies have both a larger sample and
variability in participant characteristics, hence the
magnitude, but also precision (reflected in the size of
the 95% CI) of the OR will be greater.
The OR also varies by type of internalizing disor-
der. Most clinical and epidemiologic studies showed
higher (unadjusted) OR for ODD with major depres-
sion than ODD with any anxiety disorder, a finding
that agrees with other meta-analyses of comorbidity
[5]. This finding is of interest because depression is
commonly preceded by, or comorbid with, anxiety
disorders in at least a third of cases [49], suggesting
that possibly children with anxiety rarely have ODD
in the absence of depression, or that these same
children who at one time had anxiety and ODD may
later develop major depression and ODD. As the odds
of depression and anxiety comorbidity are generally
consistently high across studies [5], such findings
may also suggest that there may be two possible tra-
jectories to depression in children: one from ODD to
major depression and one from anxiety to major
depression.
j Does this comorbidity vary between girls and boys
and at different ages?
Two studies report adjusted ORs of the association
between ODD and each internalizing disorder by
gender, but they do not compare for significant dif-
ferences in OR between genders. Maughan [39]
showed that when controlling for age and other dis-
orders, ODD was significantly associated with both
major depression and any anxiety disorder in boys,
but with anxiety disorder only in girls. Costello et al.
[17] showed that ODD and depression comorbidity
was significant for both genders but ODD and anxiety
was not, although the effect of age was not tested.
Thus, it may be that there is an interactive effect of
age and gender on this comorbidity whereby the
strength of comorbidity varies by age in girls, and not
in boys. Base rates of internalizing and oppositional
disorder do vary by gender, and by age, thus the use
of OR – controlling for age- is required to answer the
question.
j How does this comorbidity vary between
cross-sectional and longitudinal studies?
Cross-sectional studies report that approximately
25% of ODD children had internalizing disorders at
some time in childhood and adjusted ORs for inter-
nalizing disorders with ODD are significantly ele-
vated, suggesting that internalizing and oppositional
disorders may represent a commonly occurring
comorbidity. Longitudinal studies show that children
with ODD have increased risk of developing a later
comorbid internalizing disorder compared to chil-
dren without ODD or with ADHD alone [7, 36, 50].
Further, the study by August et al. [6] suggests that
different patterns of ODD over time are associated
with distinct patterns of internalizing comorbidity.
j What do these studies tell us about mechanisms
for comorbidity?
While these studies are not designed to test hypoth-
eses regarding reasons for comorbidity, the results
provide support for several mechanisms. First, as the
adjusted OR between ODD and depression – and of-
ten anxiety – is significant, this suggests a true and
specific association between ODD and internalizing
disorder. This specific association may be due to
common risk factors, such as parental history of
mood disorder or antisocial traits, parental discord,
 K. Boylan et al.
ODD and internalizing disorders
maternal hostility towards the child, child learning or
inhibitory control deficits (see review [24]), or genetic
factors [41, 47]. Such risk factors, likely in combina-
tion, may manifest in children’s behaviour as being of
both oppositional and internalizing types. Alternately,
the comorbidity may result from unique risk factors
that differ from those
causing’ pure ODD or inter-
nalizing disorders alone.
Second, as ORs of internalizing and oppositional
comorbidity were found to be significant and elevated
in both clinical, community and epidemiologic sam-
ples, it is less likely then that bias due to the clinic
referral process or to the particular measurement
tools are confounders of comorbidity estimates.
However, much remains to be studied regarding the
measurement of comorbidity. Accurate determination
of comorbidity requires that diagnostic tools contain
items that are reliable, valid and specific enough to
distinguish between possible comorbid disorders. In
this regard, the symptom of irritability is a core item
in ODD and often is a quality of depressed mood in
major depression and is a descriptor for the subjective
experience of generalized anxiety disorder. Hence,
children experiencing irritability may have a greater
likelihood of being diagnosed with comorbid ODD,
when in fact they may have a mood disorder with
secondary problems with interpersonal relations.
Speltz [50] have shown that three of eight DSM
oppositional symptoms were significantly more
common in ODD children with internalizing symp-
toms: touchy/easily annoyed, angry/resentful and
spiteful/vindictive. Simic and Fombonne [44] suggest
that children with an ICD-10 diagnosis of depressive
conduct disorder may be a group with different
clinical features than children with conduct disorder
or depressive disorder alone. These studies emphasize
the importance of considering qualitative differences
in comorbid children, which may have important
implications for the accurate measurement of behav-
iour disorder such as ODD in the presence of inter-
nalizing disorder.
Third, the studies reviewed emphasize a critical
issue about this type of comorbidity – it occurs cross-
sectionally as well as over time, and the etiologic
factors may be different in each case. Consider the
scenario in which internalizing disorders develop
several years sequential to ODD. The internalizing
disorder may result from the social and academic
failures due to the child’s oppositional behaviour [12,
13]. To support this hypothesis that ODD is a risk
factor for later internalizing disorder, it must be
shown that oppositional children did not have inter-
nalizing disorders in early life, and that experiences of
failure mediated onset of their later internalizing
disorder. Alternately, oppositionality in childhood
may be a prodrome of later internalizing disorder. In
491
other words, it may be an indicator of problems with
affect and behaviour regulation that may manifest as
mental disorders in later life [31, 48], particularly
internalizing disorder. As Speltz [50] and Lavigne [36]
have shown, internalizing and ODD comorbidity is a
common occurrence in preschoolers, thus it is un-
likely that pure oppositionality ‘‘causes’’ internalizing
symptoms in such children.
What these hypotheses suggest is that oppositional
symptoms may be risk factors for internalizing dis-
order by different processes. Identification of these
processes may be best achieved by first identifying the
trajectories by which comorbidity occurs over time.
Studies measuring concurrent comorbidity perpetuate
the myth that psychiatric comorbidity is static, while
it is clear that – at least in the case of ODD and
internalizing disorders – comorbidity changes across
development. In essence, the valid assessment of
comorbidity must incorporate longitudinal assess-
ment of the child to identify courses or trajectories of
disorder over time, and the groups of children with
such different courses compared to each other.
Limitations
While the significant ORs in epidemiologic studies
suggest there is a unique association between ODD
and internalizing disorder, it remains to be shown
how this association differs by specific type of
internalizing disorder and from the comorbidity
between internalizing disorder and CD, as links be-
tween CD and depression have been shown (See [51]
for a review). Many studies of CD and internalizing
disorder do not account for the current or past
presence of ODD in youth with CD. This is impor-
tant because most children with CD have a history of
ODD, but most children with ODD do not go on to
develop CD, thus the etiologies for comorbidity be-
tween ODD and internalizing disorders may be very
different when CD is also present. The limitation of
the majority of the studies reviewed is that they do
not specifically distinguish between the type of
anxiety disorder the child may have, or between
major depression and dythymia which limits the
type of inferences that may be drawn about specific
comorbidity effects.
If the study of comorbidity is to inform the etiol-
ogy or prognosis of psychopathology, then comor-
bidity must be assumed to occur for specific reasons.
This is challenging to study in highly comorbid dis-
orders such as depression or ODD [51]. While a
prospective study can describe differences between
children who had three disorders concurrently versus
three disorders over their lifetime, comorbidity re-
492 European Child & Adolescent Psychiatry (2007) Vol. 16, No. 8
Ó Steinkopff Verlag 2007

search must incorporate other non-symptom related
indices of risk to address the etiology of comorbidity.
Similarly, comorbidity that is measured as a cate-
gory or diagnosis may miss potentially informative
covariation in symptoms that could be derived using
a dimensional symptom count. Such information
would help identify prodromal or precursor states, as
well as important overlap in symptoms in early stages
in the evolution of child psychopathology.
Implications for clinical practice
The presence of ODD denotes lifelong mental health
issues in half of affected children. A most important
therapeutic task in diagnosing ODD is considering the
possible influence of internalizing symptoms to its
presentation and course. Two clinical scenarios can
illustrate. When children who develop ODD during
times of peak incidence for internalizing disorders
(age 7–8 and post-pubertally), the presence or emer-
gence of primary internalizing disorder should be
considered. This may be most important in girls with
later onset ODD, who are more likely to have inter-
nalizing disorders as part of the presentation. Clini-
cians should also consider whether ODD might be a
‘‘cover’’ (or even a prodrome) for internalizing
problems. When ODD is persistent over many years,
or presents in a child with other impulse or emotional
lability problems, is a mood or anxiety disorder
developing? In such instances, it might be more
therapeutically beneficial to address the difficulties
the child and family have in regulating affect and
anxiety, as there is evidence for the efficacy of CBT
and pharmacotherapy for the treatment of internal-
izing disorder [25, 36, 42, 47]. Nonetheless, as many
of these children also have had longstanding diffi-
culties with attention and behavioural regulation, it is
imperative to target the behaviour problems using
evidence-based therapy approaches.
wThe challenge for physicians is how to tell
which presentation of ODD is ‘‘only’’ ODD? More
longitudinal research is needed that assesses the
presentation of affective and anxiety symptoms in
children with ODD, as well as the differences in
response to various treatment modalities for chil-
dren and their families.
Appendix 1
Calculation of odds ratio
Anxiety No Anxiety Total
ODD A B A+B
No ODD C D C+D
Odds ratio: (Odds of event (anxiety disorder) in case group (ODD))/(Odds of
event (anxiety disorder) in control group (no ODD)) (A/(A
B))/(C/(C + D))
Calculation of effect size: Effect size: [((mean of group 1)* ) (mean of group
2)þ )/((Std. deviation (group 1)) + (Std. deviation (group 2))/2)],
*Mean of depression in oppositional children; +Mean of depression in non-
oppositional children

References
1. American Psychiatric Association
(2001) Diagnostic and Statistical Man-
ual of Mental Disorders, Fourth Edition
- Text Revisions (DSMIV-TR)
2. Anderson JC, Williams S, McGee R,
Silva PA (1987) DSM-III disorders in
preadolescent children. Prevalence in a
large sample from the general popula-
tion. Arch Gen Psychiatry 44:69–76
3. Andrews G, Henderson S, Hall W
(2001) Prevalence, comorbidity, dis-
ability and service utilisation. Overview
of the Australian National Mental
Health Survey. Br J Psychiatry 178:145–
153
4. Angold A, Costello EJ (1996) Toward
establishing an empirical basis for the
diagnosis of oppositional defiant dis-
order. J Am Acad Child Adolesc Psy-
chiatry 35:1205–1212
5. Angold A, Costello EJ, Erkanli A (1999)
Comorbidity. J Child Psychol Psychia-
try 40:57–87
6. August GJ, Realmuto GM, Joyce T,
Hektner JM (1999) Persistence and
desistance of oppositional defiant dis-
order in a community sample of chil-
dren with ADHD. J Am Acad Child
Adolesc Psychiatry 38:1262–1270
7. Biederman J, Faraone SV, Milberger S,
Jetton JG, Chen L, Mick E, Greene RW,
Russell RL (1996) Is childhood oppo-
sitional defiant disorder a precursor to
adolescent conduct disorder? Findings
from a four-year follow-up study of
children with ADHD. J Am Acad Child
Adolesc Psychiatry 35:1193–1204
8. Biederman J, Monuteaux MC, Kendrick
E, Klein KL, Faraone SV (2005) The
CBCL as a screen for psychiatric com-
orbidity in paediatric patients with
ADHD. Arch Dis Child 90:1010–1015
9. Bird HR, Canino G, Gould MS, Ribera
J, Rubio-Stipec M, Woodbury M, Hu-
ertas-Goldman S, Sesman M (1987) Use
of the child behavior checklist as a
screening instrument for epidemiolog-
ical research in child psychiatry: results
of a pilot study. J Am Acad Child
Adolesc Psychiatry 26:207–213
10. Breton JJ, Bergeron L, Valla JP, Ber-
thiaume C, St-Georges M (1998) Diag-
nostic Interview Schedule for Children
(DISC-2.25) in Quebec: reliability
findings in light of the MECA study. J
Am Acad Child Adolesc Psychiatry
37:1167–1174
11. Burke JD, Loeber R, Lahey BB, Rathouz
PJ (2005) Developmental transitions
among affective and behavioral disor-
ders in adolescent boys. J Child Psychol
Psychiatry 46:1200–1210

12. Capaldi DM, Patterson GR (1994)
Interrelated influences of contextual
factors on antisocial behavior in
childhood and adolescence for males.
Prog Exp Pers Psychopathol Res
22:165–198
13. Capaldi DM, Stoolmiller M (1999) Co-
occurrence of conduct problems and
depressive symptoms in early adoles-
cent boys: III. Prediction to young-
adult adjustment. Dev Psychopathol
11:59–84
14. Carlson CL, Tamm L, Gaub M (1997)
Gender differences in children with
ADHD, ODD, and co-occurring ADHD/
ODD identified in a school population.
J Am Acad Child Adolesc Psychiatry
36:1706–1714
15. Cohen J (1988) Statistical power anal-
ysis for the behavioural sciences. L.
Erlbaum Associates, Hillsdale, NJ
16. Cohen P, Cohen J, Kasen S, Velez CN,
Hartmark C, Johnson J, Rojas M, Brook
J, Streuning EL (1993) An epidemio-
logical study of disorders in late
childhood and adolescence–I. Age- and
gender-specific prevalence. J Child
Psychol Psychiatry 34:851–867
17. Costello EJ, Mustillo S, Erkanli A,
Keeler G, Angold A (2003) Prevalence
and development of psychiatric disor-
ders in childhood and adolescence.
Arch Gen Psychiatry 60:837–844
18. Ersan EE, Dogan O, Dogan S, Sumer H
(2004) The distribution of symptoms of
attention-deficit/hyperactivity disorder
and oppositional defiant disorder in
school age children in Turkey. Eur
Child Adolesc Psychiatry 13:354–361
19. Fleitlich-Bilyk B, Goodman R (2004)
Prevalence of child and adolescent
psychiatric disorders in southeast Bra-
zil. J Am Acad Child Adolesc Psy-
chiatry 43:727–734
20. Ford T, Goodman R, Meltzer H (2003)
The British child and adolescent mental
health survey: the prevalence of DSM-
IV disorders. J Am Acad Child Adolesc
Psychiatry 42:1203–1211
21. Gadow KD, Nolan EE (2002) Differ-
ences between preschool children with
ODD, ADHD, and ODD+ADHD
symptoms. J Child Psychol Psychiatry
43:191–201
22. Garland EJ, Garland OM (2001) Cor-
relation between anxiety and opposi-
tionality in a children’s mood and
anxiety disorder clinic. Can J Psychia-
try 46:953–958
23. Greene RW, Doyle AE (1999) Toward a
transactional conceptualization of
oppositional defiant disorder: implica-
tions for assessment and treatment.
Clin Child Fam Psychol Rev 2:129–148
K. Boylan et al.
ODD and internalizing disorders
24. Hinshaw SP (1987) On the distinction
between attentional deficits/hyperac-
tivity and conduct problems/aggression
in child psychopathology. Psychol Bull
101:443–463
25. James A, Soler A, Weatherall R (2005)
Cognitive behavioural therapy for
anxiety disorders in children and ado-
lescents. Cochrane Database Syst Rev
CD004690
26. Kadesjo B, Gillberg C (2001) The
comorbidity of ADHD in the general
population of Swedish school-age chil-
dren. J Child Psychol Psychiatry
42:487–492
27. Kashani JH, Beck NC, Hoeper EW,
Fallahi C, Corcoran CM, McAllister JA,
Rosenberg TK, Reid JC (1987) Psychi-
atric disorders in a community sample
of adolescents. Am J Psychiatry
144:584–589
28. Keenan K, Wakschlag LS (2004) Are
oppositional defiant and conduct dis-
order symptoms normative behaviors
in preschoolers? A comparison of re-
ferred and nonreferred children. Am J
Psychiatry 161:356–358
29. Kim-Cohen J, Caspi A, Moffit TE,
Harrington H, Milne BJ, Poulton R
(2003) Prior juvenile diagnoses in
adults with mental disorder: develop-
mental follow-back of a prospective-
longitudinal cohort. Arch Gen Psychi-
atry 60:709–717
30. Kuhne M, Schachar R, Tannock R
(1997) Impact of comorbid opposi-
tional or conduct problems on atten-
tion-deficit hyperactivity disorder. J
Am Acad Child Adolesc Psychiatry
36:1715–1725
31. Lahey BB, Goodman SH, Waldman ID,
Bird H, Canino G, Jensen P, Regier D,
Leaf PJ, Gordon R, Applegate B (1999)
Relation of age of onset to the type and
severity of child and adolescent con-
duct problems. J Abnorm Child Psy-
chol 27:247–260
32. Lahey BB, Schwab-Stone M, Goodman
SH, Waldman ID, Canino G, Rathouz
PJ, Miller TL, Dennis KD, Bird H, Jen-
sen PS (2000) Age and gender differ-
ences in oppositional behavior and
conduct problems: a cross-sectional
household study of middle childhood
and adolescence. J Abnorm Psychol
109:488–503
33. Lavigne JV, Gibbons RD, Christoffel
KK, Arend R, Rosenbaum D, Binns H,
Dawson N, Sobel H, Isaacs C (1996)
Prevalence rates and correlates of psy-
chiatric disorders among preschool
children. J Am Acad Child Adolesc
Psychiatry 35:204–214
493
34. Lavigne JV, Cicchetti C, Gibbons RD,
Binns HJ, Larsen L, DeVito C (2001)
Oppositional defiant disorder with on-
set in preschool years: longitudinal
stability and pathways to other disor-
ders. J Am Acad Child Adolesc Psy-
chiatry 40:1393–1400
35. Loeber R, Burke JD, Lahey BB, Winters
A, Zera M (2000) Oppositional defiant
and conduct disorder: a review of the
past 10 years, part I. J Am Acad Child
Adolesc Psychiatry 39:1468–1484
36. March J, Silva S, Petrycki S, Curry J,
Wells K, Fairbank J, Burns B, Domino
M, McNulty S, Vitiello B, Severe J
(2004) Fluoxetine, cognitive-behavioral
therapy, and their combination for
adolescents with depression: Treatment
for Adolescents With Depression Study
(TADS) randomized controlled trial.
JAMA 292:807–820
37. Maughan B, Rowe R, Messer J, Good-
man R, Meltzer H (2004) Conduct dis-
order and oppositional defiant disorder
in a national sample: developmental
epidemiology. J Child Psychol Psychi-
atry 45:609–621
38. McGee R, Feehan M, Williams S, Par-
tridge F, Silva PA, Kelly J (1990) DSM-
III disorders in a large sample of ado-
lescents. J Am Acad Child Adolesc
Psychiatry 29:611–619
39. Neuman RJ, Sitdhiraksa N, Reich W, Ji
TH, Joyner CA, Sun LW, Todd RD
(2005) Estimation of prevalence of
DSM-IV and latent class-defined
ADHD subtypes in a population-based
sample of child and adolescent twins.
Twin Res Hum Genet 8:392–401
40. Rowe R, Maughan B, Costello EJ, An-
gold A (2005) Defining oppositional
defiant disorder. J Child Psychol Psy-
chiatry 46:1309–1316
41. Schachar R, Wachsmuth R (1990)
Oppositional disorder in children: a
validation study comparing conduct
disorder, oppositional disorder and
normal control children. J Child Psy-
chol Psychiatry 31:1089–1102
42. Seidel L, Walkup JT (2006) Selective
serotonin reuptake inhibitor use in the
treatment of the pediatric non-obses-
sive-compulsive disorder anxiety dis-
orders. J Child Adolesc
Psychopharmacol 16:171–179
43. Rey JM (1994) Comorbidity between
disruptive bahaviour disorders and
depression in referred adolescents.
Aust N Z J Psychiatry 28:106–113
44. Simic M, Fombonne E (2001) Depres-
sive conduct disorder: symptom pat-
terns and correlates in referred
children and adolescents. J Affect Dis-
ord 62:175–185
494
45. Simonoff E, Pickles A, Meyer JM, Sil-
berg JL, Maes HH, Loeber R, Rutter M,
Hewitt JK, Eaves LJ (1997) The Virginia
Twin Study of adolescent behavioral
development. Influences of age, sex,
and impairment on rates of disorder.
Arch Gen Psychiatry 54:801–808
46. Speltz ML, McClellan J, DeKlyen M,
Jones K (1999) Preschool boys with
oppositional defiant disorder: clinical
presentation and diagnostic change. J
Am Acad Child Adolesc Psychiatry
38:833–845
European Child & Adolescent Psychiatry (2007) Vol. 16, No. 8
Ó Steinkopff Verlag 2007
47. Stevenson J, Asherson P, Hay D, Levy
F, Swanson J, Thapar A, Willcutt E
(2005) Characterizing the ADHD phe-
notype for genetic studies. Dev Sci
8:115–121
48. Weissman MM (2006) Recent advances
in depression across the generations.
Epidemiol Psichiatr Soc 15:16–19
49. Wittchen HU, Nelson CB, Lachner G
(1998) Prevalence of mental disorders
and psychosocial impairments in ado-
lescents and young adults. Psychol Med
28:109–126
50. Yorbik O, Birmaher B, Axelson D,
Williamson DE, Ryan ND (2004) Clin-
ical characteristics of depressive
symptoms in children and adolescents
with major depressive disorder. J Clin
Psychiatry 65:1654–1659
51. Zoccolillo M (1992) Co-occurrence of
conduct disorder and its adult out-
comes with depressive and anxiety
disorders: a review. J Am Acad Child
Adolesc Psychiatry 31:547–556