Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12

Contents lists available at ScienceDirect

Journal of Obsessive-Compulsive and Related Disorders

journal homepage: www.elsevier.com/locate/jocrd

Content overlap between youth OCD scales: Heterogeneity among symptoms T

probed and implications
Rachel Visontaya,
⁎,1
, Matthew Sunderlanda,1, Jessica Grishamb, Tim Sladea,1
a
NHMRC Centre of Research Excellence in Mental Health and Substance Use, National Drug and Alcohol Research Centre, University of New South Wales, Sydney,
Australia
b
School of Psychology, University of New South Wales, Sydney, NSW 2052, Australia

ARTICLE INFO ABSTRACT

Keywords: Assessing obsessive compulsive disorder (OCD) in paediatric populations is complex, and a number of scales
Obsessive-compulsive disorder have been developed or validated for this purpose. As is the case with instruments measuring other mental
OCD health disorders, it is often assumed that youth OCD scales assess the same construct and can be used inter-
Measurement changeably. However, the extent to which this is true is unknown. One way to assess this – beyond calculating
Assessment
convergent validity – is to measure the extent of content overlap between scales. In the current paper, we apply a
Child
Adolescent
recently-developed method of content overlap analysis to a set of freely-available, self-report instruments for
OCD measurement that were developed for or validated in paediatric populations. We found a high level of
heterogeneity among scales purportedly assessing the same construct. Implications for clinical practice and
research are discussed, as well as what further research is required.

1. Introduction DSM-5 (American Psychiatric Association, 2013) notes that presenta-

Obsessive-compulsive disorder (OCD) is characterised by the pre-
sence of obsessions and/or compulsions, which impact on daily func-
tioning or cause psychological distress (American Psychiatric
Association, 2013). However, there are complications in the disorder's
diagnosis and measurement given the heterogeneous nature of OCD and
the various combinations of potential symptoms present (Lewin, Park,
& Storch, 2013). Additionally, psychopathologies such as anxiety and
autism spectrum disorders may present similarly to OCD (Berman &
Abramowitz, 2010; Lewin & Piacentini, 2010), and OCD is highly co-
morbid with these and other disorders, including tic, depressive, at-
tention-deficit/hyperactivity, and behavioural disorders (Lewin et al.,
2013; Storch et al., 2008).
The measurement of the disorder in youth populations has its own
complexities, especially as children often lack insight into their OCD
(Lewin et al., 2013). In childhood, prevalence estimates of OCD range
from 0.25% (5–15 years; Heyman et al., 2003) to 2.7% (9–17 years;
Rapoport et al., 2000), and up to 4% in adolescents specifically (18
years; Douglass, Moffitt, Dar, McGee, & Silva, 1995). The disorder is
associated with significant functional impairment in those aged 5–17
(Piacentini, Bergman, Keller, & McCracken, 2003). Importantly, the
tion of OCD in youth may differ to that in adults. For example, the
content of obsessions and compulsions may vary, children may struggle
to articulate obsessions and so they may be less obvious than observable
compulsions, and younger children may find it difficult to describe the
aims of their behaviours or thoughts (American Psychiatric Association,
2013). There are also disorder characteristics specific to children, e.g.,
an especially high prevalence of family accommodation, i.e., involving
relatives in rituals (Lewin et al., 2013; Peris et al., 2008). As such, in
addition to having strong psychometric properties, it is important that
OCD scales administered to paediatric populations have been specifi-
cally created for them and are developmentally appropriate or have at
least been validated in youth samples.
There are several scales available for use with paediatric popula-
tions, including both those specifically assessing OCD, and those that
examine OCD among other psychopathology. A recent review identified
14 OCD-specific measures whose psychometric properties had been
assessed in youth samples (Iniesta-Sepúlveda, Rosa-Alcázar, Rosa-
Alcázar, & Storch, 2014). As is often the case with instruments used to
assess specific mental health disorders, it is generally assumed that
these OCD scales measure the same construct, and as such, scales are
used interchangeably and results from one compared against another.

Corresponding author.
⁎

E-mail addresses: r.visontay@unsw.edu.au (R. Visontay), matthews@unsw.edu.au (M. Sunderland), jessicag@unsw.edu.au (J. Grisham),
tims@unsw.edu.au (T. Slade).
1
Postal Address: National Drug and Alcohol Research Centre, Building R1, 22–33 King St, Randwick NSW 2031, Australia.

https://doi.org/10.1016/j.jocrd.2018.10.005
Received 5 September 2018; Received in revised form 29 October 2018; Accepted 31 October 2018
Available online 02 November 2018
2211-3649/ © 2018 Elsevier Inc. All rights reserved.
R. Visontay et al.
Whether the former is true, and the latter a sound practice, is important
to verify. If scales do not assess the same construct, this hinders our
ability to draw conclusions about epidemiological data and interven-
tion results as they relate to a given disorder. Rather, we may only be
able to speak in terms of scores on a given instrument (Fried, 2017). A
scale's symptom coverage also has implications for the ability of that
instrument to discern disorder subtypes, which is of particular concern
for OCD given a presentation heterogeneity that is more marked than in
other disorders (Storch et al., 2010). Considering evidence in adult
populations that certain symptom profiles may respond differently to
therapy (Sookman, Abramowitz, Calamari, Wilhelm, & McKay, 2005;
Williams, Mugno, Franklin, & Faber, 2013), and similar findings based
on limited research in youth populations as well (Storch et al., 2008),
the ability of an instrument to identify symptom profiles/subtypes is
key to selecting appropriate treatments.
There are several reasons to suspect that youth OCD scales may not
measure the same construct – scales have been created by different
research groups, for different research or clinical purposes, and with
different populations in mind. Additionally, some assessments are
biased towards obsessions over compulsions (or vice versa), and ex-
isting scales vary markedly in length. One way to assess whether scales
are measuring the same construct is to calculate convergent validity,
i.e., how actual scores on different tests correlate. There is mixed evi-
dence for the convergent validity of youth OCD assessments, with some
scales demonstrating strong convergent validity, some weak, and others
lacking in research (Iniesta-Sepúlveda et al., 2014).
However, it is possible that despite two scales displaying high
convergent validity, they may not actually be assessing the same con-
struct. This could be due to high rates of comorbidity (and so people
that score highly on a scale measuring construct A may also score
highly on a scale measuring construct B). More probable in the case of
youth OCD, scales may be assessing different aspects of the same
broader construct, thus not overlapping well in terms of item content
but displaying high convergent validity due to inter-relatedness of
constructs or some common underlying construct. To explore this
possibility, a complementary method of comparing instruments is to
assess the overlap of item content between the scales themselves, ir-
respective of actual human performance on those scales. Developing a
method of content overlap analysis that utilises the Jaccard similarity
coefficient, Fried (2017) assessed the overlap between seven widely-
used depression scales and found considerable heterogeneity among the
instruments, concluding that results from depression research may only
be interpretable in terms of the specific scale used. He found a moderate
correlation between number of symptoms captured by a scale and how
well it overlapped with other scales, suggesting longer scales were more
representative. To the best of the authors’ knowledge, his is the only
study to implement such analysis.
This paper aims to apply the method outlined by Fried (2017) to a
set of freely-available, self-report instruments that exclusively measure
OCD and were developed for or validated in paediatric populations. Our
focus here is on content overlap of items that probe the presence of
specific symptoms, as opposed to items assessing symptom severity or
functional impairment. For the reasons outlined earlier, we expect to
find some level of heterogeneity among youth OCD scales.
2. Methods
In the current study, we applied Fried's (2017) method of assessing
content overlap across multiple scales that purportedly measure the
same construct, to youth OCD scales, comprising several steps. The
process began with a systematic approach to scale selection, followed
by item extraction. Stage two consisted of content analysis, which in-
volved collapsing similar items within scales, and comparing collapsed
items across scales. Item characteristics (unique or not, and compound
or specific) were noted, and umbrella symptom names were generated.
In the final stage, Jaccard analyses were conducted to estimate the level
7
Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12
of overlap between individual scales, as well as the mean overlap for all
scales.
2.1. Selecting OCD scales and extracting items
The OCD scales were obtained from a systematic review of youth
self-report instruments measuring five different disorders (including
OCD). This review involved a systematic search for scales designed
for or validated in child or adolescent populations, searching for
child/adolescent adaptations or validations of adult scales identified
in a previous review (Batterham et al., 2015), and supplementation
with additional scales identified in recent review papers. Systematic
searches of Medline and PsyINFO were conducted in March 2018,
involving a search strategy combining three groups of MeSH terms
and free text keywords, largely adapted from the Batterham et al.
(2015). These groups of terms were: (1) disorder-specific terms
combined with ‘OR’ operators; (2) terms related to psychometrics,
scales, measurement and self-report combined with ‘OR’ operators;
and (3) age-specific terms to capture papers involving children and
adolescents combined with ‘OR’ operators. Inclusion criteria were
that the papers reported on the development, evaluation or use of
scales which were designed, adapted or validated for youth popula-
tions to assess the presence and/or severity of at least one of the
target disorders, in self-report format, freely available, and in
English.
The review identified 11 self-report scales developed for or vali-
dated in youth populations which contained items assessing OCD. Of
these, seven scales exclusively assessed OCD and probed both obses-
sions and compulsions. These were: the Children's Florida Obsessive
Compulsive Inventory (C-FOCI; Storch et al., 2009), the Children's Yale-
Brown Obsessive Compulsive Scale Symptom Checklist (CY-BOCS;
Storch et al., 2006), the Leyton Obsessional Inventory-Child Version
(LOI-CV; Berg, Rapoport, & Flament, 1986), the Children's Obsessional
Compulsive Inventory-Revised-Self Report (ChOCI-R-S; Uher, Heyman,
Turner, & Shafran, 2008), the Obsessive Compulsive Inventory-Child
Version (OCI-CV; Foa et al., 2010), the OCD Family Functioning Scale
(OFF; Stewart et al., 2011), and the Short OCD Screener (SOCS; Uher,
Heyman, Mortimore, Frampton, & Goodman, 2007). Combined, these
scales yielded 238 items. Given the focus of this study on comparing the
content of obsessions and compulsions, items targeting functional im-
pairment and distress caused by obsessions and compulsions were re-
moved for the purposes of analysis. This left 206 symptom-focused
items.
2.2. Collapsing items, generating umbrella symptoms
We then undertook content analysis. Given that we already ex-
pected substantial heterogeneity among scales, we were conservative
in this process – erring on the side of underestimating heterogeneity
where possible. The first step was to decide whether a given item
probed an obsession, compulsion or both/neither. Then, we identified
items within each scale that were similarly worded or probed the same
symptom, and collapsed these items into a more general item. For
example, in the CY-BOCS, ‘Excessive concern with environmental
contaminants (e.g., asbestos or radioactive substances)’ and ‘Excessive
concern with contamination from household items (e.g., cleaners,
solvents)’ were collapsed into a broader contamination/germs obses-
sion. This is a necessarily subjective process, and so we erred on the
side of collapsing more items together, as to minimise heterogeneity
when later making comparisons between scales. We also collapsed
items across content or various manifestation, e.g. any item assessing
intrusive thoughts (be they images, sounds, words or numbers) of a
violent or non-violent nature were collapsed into one. Similarly, items
that probe for checking directed at different targets, such as checking
one's body or checking for disasters, were collapsed into one checking
compulsion.
R. Visontay et al. Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12

Next, we noted any items which assessed multiple symptoms (i.e., Table 1
‘compound items’). For example, in the OFF scale, the item probing Number of obsession symptoms (/32) that appear across combinations of
‘Contamination’ was deemed a compound item, in that it could receive scales.
endorsement from a responder with either a contamination-related Scales Symptoms %
obsession or contamination-related compulsion. Similarly, in the SOCS,
‘Does your mind often make you do things – such as checking or 1 18 56
2 7 22
touching things or counting things’ goes to separate checking, touching
3 4 13
and counting compulsions. Items which were deemed to only probe a 4 1 3
single symptom were counted as specific items. If no other item in the 5 2 6
scale had assessed any of the symptoms contained in a compound item, 6 –
7 –
then that item was expanded into two separate items (the opposite of
the collapsing process). The ‘Symptoms Captured’ column in Table 3
reflects the number of symptoms captured after the collapsing and ex-
Table 2
panding process for each scale.
Number of compulsion symptoms (/21) that appear across combinations
We then compared items across scales to determine the extent of of scales.
content overlap. We determined which symptoms were probed in
Scales Symptoms %
multiple scales, and which were uniquely probed by just one scale.
Determining if differently worded items or collapsed items are probing 1 8 38
the same symptom is subjective, so again, we adopted a conservative 2 2 10
approach so as not to overestimate scale heterogeneity. That is, we tried 3 3 14
as much as possible to match items across scales, rather than deem that 4 4 19
5 1 5
they were assessing disparate symptoms. We then generated names for 6 1 5
the umbrella symptoms which captured the similar items across scales. 7 2 10
Finally, two tables – one for obsessions and one for compulsions –
listing all umbrella symptoms and all scales were constructed. For each
symptom, scales were awarded a 2 (contains an item specifically Tables 1 and 2 display the number of symptoms shared by increasing
probing the symptom), a 1 (contains a compound item that probes the numbers of scales. Of note, 18 of the 32 obsessions appear in just one
symptom) or a 0 (does not contain an item probing the symptom). scale, while two obsessions (contamination/germ obsessions and in-
These tables are provided in the Supplementary material. trusive thoughts/images) appear in five of seven scales. Eight of the 21
compulsions (39%) appear in just one scale, while cleaning compul-
2.3. Jaccard analysis of overlap sions and checking compulsions were featured in all seven scales. The
one symptom that could not be counted as an obsession or compulsion
We then performed a Jaccard analysis of the overlap between in- was related to giving poor class presentations despite preparation,
dividual scales. First, we modified the tables created in the previous step found in one scale (the LOI-CV). See Figs. 1 and 2 for a graphical pre-
so that they no longer differentiated between symptoms probed by spe- sentation of scale overlap.
cific and compound items (only between symptoms probed and not
probed). Using these modified tables, we calculated the Jaccard simi-
Obsessions Compulsions
larity coefficient, which reflects how similar two given sets are. The
1. Contamination/germs obsession 1. Cleaning
Jaccard coefficient is calculated for each pairwise combination of scales 2. Need for order/arrangement 2. Checking
by taking the number of shared symptoms and dividing by the total 3. Intrusive thoughts/images 3. Counting/arranging
number of symptoms across both scales (shared and unique). Therefore, 4. Disaster obsession 4. Repeating
this coefficient, ranging from 0 to 1, represents the proportion of shared 5. Causing harm to others obsession 5. Touching
6. Causing harm to self obsession 6. Avoiding unlucky numbers/words/colours
symptoms among the total number of symptoms. In the absence of ex- 7. Losing things obsession 7. Confession/reassurance seeking
isting rules for interpretation of the Jaccard coefficient, we were con- 8. Fear something bad will happen 8. Hoarding/saving
sistent with the rule used by Fried (2017): 0.00–0.19 very weak, to self 9. Superstitious physical behaviour
0.20–0.39 weak, 0.40–0.59 moderate, 0.60–0.79 strong, 0.80–1 very 9. Fear something bad will happen 10. Involving others in rituals
to others 11. Preventing harm (non-checking)
strong. We then calculated the mean overlap per scale and total mean
10. Fear will act without control 12. Ritualised eating
overlap between all scales. To shed light on what scale characteristics 11. Worry about discarding important 13. Excessive list making
were associated with low or high overlap, we then correlated each mean things 14. Doing things until ‘just right’
overlap with scale length (number of symptoms probed by that scale), 12. Health obsession 15. Trouble completing schoolwork or chores
and additionally, with percentage of scale items that were unique. 13. Religious obsession due to repetition
14. Morality obsession 16. Saying words/numbers over and over
As scales focused differentially on obsessions/compulsions, we 15. Lucky or unlucky numbers/ 17. Punctuality
conducted analyses separately for obsession and compulsion symptoms words/colours 18. Ordered dressing
(the one symptom which was neither an obsession nor a compulsion 16. Sexual obsession 19. Acting on lucky numbers
was excluded from the Jaccard analysis). Jaccard analyses were con- 17. Fear of doing something embar- 20. Counting while doing other things
rassing 21. Allowance/money compulsions
ducted in R, and the scripts for these analyses and for figure con-
18. Need to know or remember Other
struction (both adapted from Fried (2017)) are available in the 19. Fear of not saying right thing 22. Give poor class presentations despite
Supplementary material. 20. Bothered by sounds planning
21. Discomfort unless things done
3. Results ‘just right’
22. Feel must do things despite not
really having to
3.1. Symptoms and scale composition 23. Fear for family
24. Dislike touch
The process of collapsing and expanding items, and then generating 25. Worry about dangerous objects
26. Perfection obsession
umbrella symptoms, yielded 54 total umbrella symptoms across scales,
comprising 32 obsessions, 21 compulsions, and 1 other symptom.

8
R. Visontay et al. Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12

27. Doubt whether completed task Table 3

post-completion Scale composition.
28. Rumination
29. Obsessive slowness Scale Symptoms Unique Specific Compound
30. Thought control obsession captured symptoms (%) items (%) items (%)
31. Indecision (obsession
32. Sanity obsession symptoms)

C-FOCI 13 (6) 1 (8) 13 (100) 0 (0)

CY-BOCS 32 (18) 14 (45) 30 (94) 2 (6)
LOI-CV 27 (12) 9 (33) 26 (96) 1 (4)
Twenty-seven of the 54 umbrella symptoms (50%) were unique to OCI-CV 11 (5) 2 (18) 11 (100) 0 (0)
just one scale. The CY-BOCS had the highest percentage of unique OFF 14 (8) 1 (7) 12 (86) 2 (14)
symptoms (symptoms captured by no other scale) as a percentage of SOCS 7 (1) 0 (0) 4 (57) 3 (43)
ChOCI-R-S 16 (7) 1 (6) 15 (94) 1 (6)
total symptoms captured by the scale, as well as the greatest number
of symptoms captured overall. The SOCS had no unique symptoms, Note: Obsessive, compulsive and other symptoms are all included. ‘Specific
but it captured the fewest symptoms overall. The majority of unique Items (%)’ reflects the percentage of ‘Symptoms Captured’ by the scale that
symptoms were obsessive symptoms (18; 67%), eight (30%) were were specifically probed by an item, while ‘Compound Items (%)’ reflects the
compulsions and one was ‘other’ (4%). There were no great differ- percentage probed by an item that captured more than one symptom (implicitly
ences in terms of percentage of symptoms probed as part of a specific or explicitly). ‘Compound Items (%)’ does not capture items in original scales
or compound item, although almost half of the symptoms captured that were later collapsed with other similarly-worded items on all included
by the SOCS were done so via compound items. Scale composition is symptoms e.g., in LOI-CV, “Do you move or talk in just a special way, to avoid
bad luck?” probed both superstitious physical behaviour and avoiding unlucky
displayed in Table 3.
words, but was collapsed with both other superstitious physical behaviour
items and other avoiding unlucky words items in arriving at the final adjusted
scale. Similarly, ‘Specific Items (%)’ captures only the items included in the
final adjusted scale, after any expanding or collapsing of original items.

3.2. Jaccard analysis

Fig. 1. Obsession symptoms captured by each scale. An empty circle indicates
that the symptom was only probed as part of a compound item, while a
filled-in circle indicates that an item in that scale specifically assessed the
symptom.
3.2.1. Obsessions
The mean overlap between all scales on obsession symptoms was
0.14 (very weak). As it only contributed one item, we also ran the
analysis excluding the SOCS. Mean overlap was still very weak (0.19).
Overlap between pairs of scales, as well as mean overlap for each scale,
is displayed in Table 4. Of note, the SOCS had the lowest mean overlap
of any scale at 0.03, which is explained by the fact that it only con-
tributed one obsession item, while the C-FOCI and LOI-CV had the
highest at 0.19, followed by the OFF at 0.18. The C-FOCI/OCI-CV had
the highest pairwise overlap at 0.38, while the SOCS had 0% overlap
with three of the six other scales.
The correlation between number of symptoms captured and mean
overlap was 0.43 (medium) suggesting that, for obsession symptoms,
scale length played some role in determining overlap.
Additionally, we correlated the percentage of obsession symptoms
per scale that were unique to that scale, with the mean overlap for the
scale (excluding the SOCS). The correlation was −0.08, suggesting
little relationship between the proportion of symptoms uniquely probed
by a particular scale and how well its content overlaps with other
scales.

3.2.2. Compulsions
The mean overlap between all scales on compulsion symptoms was
0.39 (weak). Overlap between pairs of scales, as well as mean overlap
for each scale, is displayed in Table 5. Of note, the OCI-CV had the
lowest mean overlap of any scale at 0.29, while the ChOCI-R-S had the

Table 4
Overlap values from Jaccard analysis for obsession symptoms.
C-FOCI CY-BOCS LOI-CV OCI-CV OFF SOCS ChOCI-R-S

C-FOCI 1.00 0.14 0.29 0.38 0.17 0.00 0.17

CY-BOCS 0.14 1.00 0.15 0.10 0.30 0.00 0.13
LOI-CV 0.29 0.15 1.00 0.21 0.25 0.08 0.18
OCI-CV 0.38 0.10 0.21 1.00 0.18 0.00 0.08
OFF 0.17 0.30 0.25 0.18 1.00 0.13 0.07
Fig. 2. Compulsion (and ‘other’) symptoms captured by each scale. An empty circle SOCS 0.00 0.00 0.08 0.00 0.13 1.00 0.00
indicates that the symptom was only probed as part of a compound item, while a ChOCI-R-S 0.17 0.13 0.18 0.08 0.07 0.00 1.00
Mean 0.19 0.14 0.19 0.16 0.18 0.03 0.10
filled-in circle indicates that an item in that scale specifically assessed the symptom.

9
R. Visontay et al. Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12

Table 5 4.2. Individual scale performance

Overlap values from Jaccard analysis for compulsion symptoms.
C-FOCI CY-BOCS LOI-CV OCI-CV OFF SOCS ChOCI-R-S The OFF scale displayed the second highest mean overlap for both
obsessions and compulsions. It had a very low percentage of unique
C-FOCI 1.00 0.50 0.31 0.18 0.44 0.44 0.60 symptoms, as well as a relatively high proportion of symptoms probed
CY-BOCS 0.50 1.00 0.40 0.18 0.43 0.33 0.44
by compound items. This high overlap may be explained by the very
LOI-CV 0.31 0.40 1.00 0.33 0.43 0.33 0.35
OCI-CV 0.18 0.18 0.33 1.00 0.33 0.33 0.36
general, concise phrasing of items in the OFF scale e.g., ‘Checking’ or
OFF 0.44 0.43 0.43 0.33 1.00 0.50 0.36 ‘Contamination’, which matched up well to the umbrella symptom
SOCS 0.44 0.33 0.33 0.33 0.50 1.00 0.50 terms we developed in carrying out the content analysis. No scales
ChOCI-R-S 0.60 0.44 0.35 0.36 0.36 0.50 1.00 stood out as particularly poor performers, although the OCI-CV clearly
Mean 0.41 0.38 0.36 0.29 0.42 0.41 0.44
had the least overlap with other scales for compulsion symptoms
(bearing in mind the small number of compulsion symptoms that it
probed).
highest at 0.44, followed by the OFF at 0.42. The C-FOCI/ChOCI-R-S Interestingly, while the CY-BOCS is the most widely used scale in
displayed the highest pairwise overlap at 0.60. The C-FOCI/OCI-CV and paediatric OCD research (Freeman, Flessner, & Garcia, 2011; Watson &
CY-BOCS/OCI-CV displayed the lowest pairwise overlap at 0.18. Rees, 2008), and is considered the “gold standard” in the field (Iniesta-
The correlation between number of symptoms captured and mean Sepúlveda et al., 2014), it performed somewhere in the middle in terms
overlap was −0.09, suggesting that, for compulsion symptoms, scale of overlap. This can be somewhat explained by it having the greatest
length did not determine the extent of scale overlap. The correlation length and a high number of unique symptoms probed. Here, not
between the percentage of compulsion symptoms per scale that were having the highest mean overlap is not necessarily a weakness, as it
unique to that scale, with the mean overlap for the scale, was −0.20. may be a result of the instrument being the most comprehensive and
This suggests only a weak relationship between increasing proportion varied (and as such, capturing a broader range of the construct). That
of unique symptoms probed by a scale, and lower overlap with other said, maximum comprehensiveness is not necessarily preferable, as the
scales. varying prevalence of symptoms means some items are more valuable
diagnostically than others, and because researchers/clinicians may
sometimes want to use scales with a narrower focus.
4. Discussion
4.3. Dissociations with convergent validity
Symptom overlap between youth OCD scales was quite low: 0.14
and 0.39 for obsession and compulsion symptoms respectively. Fifty-six Although convergent validity analyses are limited – part of the
percent of obsession symptoms, and 38% of compulsion symptoms, motivation for our choice of methodology – one advantage of using
were unique to just one scale. The results have important implications convergent validity over content overlap data is that it obviates the
for how we use and interpret epidemiological and intervention data need for subjective judgements about whether two items are assessing
that employ such instruments. Unlike Fried (2017), we found almost no the same symptoms. Additionally, convergent validity values are not as
(compulsions) and medium (obsessions) relationships between number affected by mismatches in scale lengths, which had a particularly stark
of symptoms probed by a scale and mean overlap, and additionally, effect in our analysis for the SOCS. However, it might also be argued
very weak relationships between proportion of a scale's assessed that the brevity of a scale such as the SOCS, whose scope is intentionally
symptoms that are unique and mean overlap for both obsessions and limited due to its design as a screening instrument, should be reflected
compulsions. That is, these scale characteristics are not particularly in some way in a measure of the extent to which is assesses the same
(and at best, moderately) associated with scale overlap or lack thereof. construct as another scale.
That said, if we have both convergent validity and content overlap
data available for comparison, this can further elucidate the similarity
4.1. Obsessions versus compulsions of two scales (or lack thereof). As previously mentioned, it is possible
for a set of two scales to have dissociations between convergent validity
One of the most notable findings of this analysis was the greater and content overlap, despite both ostensibly being measures of the
overlap between compulsion symptoms than between obsession symp- extent to which the two scales measure the same construct. If con-
toms. There were far more distinct obsession symptoms than compul- vergent validity is strong, then respondents are scoring similarly on the
sions, and a larger percentage of obsession symptoms were unique to two scales, generally because those scales measure the same broader
just one scale. Removing the SOCS (which only contributed one construct. However, if convergent validity is strong but content overlap
symptom) from the obsession analysis did not make a substantial im- is low, this would suggest that the scales are measuring different aspects
provement to the mean overlap between obsession symptoms. Given of that construct. While convergent validity scores have been calculated
some indication that obsessions are harder to identify than compulsions in previous studies for several pairs of the scales examined here, many
in children (American Psychiatric Association, 2013), it could be ar- of these use clinician-rated versions of scales, or focus on symptom
gued that the larger number of obsession symptoms is justified in giving severity components. However, there are a few relevant data points we
respondents more options that may better reflect their specific obses- can consider. For example, dissociations are evident between con-
sions. However, high heterogeneity means that any one scale provides vergent validity (0.61–0.65) and content overlap (0.00 and 0.44 for
only a portion of this range, limiting any benefit. obsessions and compulsions respectively) for the SOCS and the C-FOCI
It is also important to note that, from scale to scale, there is con- symptom checklist (Piqueras et al., 2015). Using values from the same
siderable variation in the focus placed on probing obsessions compared paper, we can see a similar divergence when it comes to convergent
with compulsions. Some instruments (not included in these analyses), validity for the SOCS and OCI-CV total score (0.72–0.80) and content
focus exclusively on one of the two kinds of symptoms; others, such as overlap (0.00 and 0.33 for obsessions and compulsions respectively).
the SOCS, are heavily biased in their focus; while others still, such as There is clearly a need for collecting further convergent validity
the CY-BOCS, are more balanced. This is problematic, given most data, and the lack of existing data means we are limited in our ability to
children have both obsessive and compulsive symptoms (American draw conclusions about the dissociations between convergent validity
Psychiatric Association, 2013), and so the use of unbalanced scales may and content overlap. However, if these dissociations bear out in future
not accurately capture the symptomatology of respondents. research, this would support the idea that scales are in fact assessing

10
R. Visontay et al.
OCD more broadly, but differ on which aspects of the construct they
probe, or how comprehensive they are.
4.4. Strengths and limitations
This paper is the first to implement a content overlap methodology
to assess whether youth OCD scales are measuring the same construct,
as opposed to using convergent validity analyses. Our analysis offers
information specific to items assessing the presence of particular OCD
symptoms, and we provide key descriptive characteristics of these in-
struments, such as the percentage of unique items, as well as content
overlap values.
We took many steps to be conservative in underestimating hetero-
geneity, and to mitigate subjectivity. These measures included ana-
lysing obsession and compulsion symptoms separately, as well as erring
on the side of collapsing items together and finding matches across
scales (rather than classifying symptoms as distinct). Despite this, the
greatest limitation of this study was the subjective nature of the content
analysis. It is possible that the collapsing and expanding stage would
have produced slightly different results if performed by another re-
search group. Similarly, the interpretation of the Jaccard coefficients
was subjective, given no established rules for interpretation, which is
why we also provided the raw overlap values.
Finally, there are limitations related to scale selection. Firstly, given
variation in symptom profiles between younger and older youth (Selles,
Storch, & Lewin, 2014), heterogeneity might be partly attributable to
comparing scales that were developed for slightly different age ranges.
More generally, our results cannot be extended to all scales assessing
OCD in youth, given we only analysed self-report scales that were free
and accessible, and that did not also measure other psychopathology.
However, these factors also played a role in preventing the inflation of
heterogeneity which would otherwise occur due to differences between
self and clinician-administered scales, or differences between scales
that measure OCD specifically and those that measure psychopathology
more broadly.
4.5. Implications and future directions
Given the considerable heterogeneity demonstrated, it is clear that
scales cannot be used interchangeably, and comparisons of results ob-
tained from different instruments must be made with caution. As
mentioned, the dissociation between convergent validity and content
overlap values may suggest that while youth OCD scales generally
measure the same broader construct, they might have different scopes
within that construct, or may be more or less comprehensive than one
another. As such, low content overlap does not necessarily mean a scale
is ‘worse’ than one with higher overlap. However, researchers and
clinicians should be careful in selecting scales to assess OCD sympto-
matology, being mindful of the specific characteristics of each instru-
ment. Due to its relatively high overlap, the OFF scale may be a good
choice for those wishing to make comparison to results based on other
instruments, while the CY-BOCS is a strong option for those wanting a
comprehensive measure allowing for the identification of symptom
profiles/subtypes.
One solution to this problem is the approach offered by item
banking. Via a process of gathering pools of items from existing scales,
item standardisation, content validity testing, consumer and expert
evaluation, and refinement via item response theory (IRT), item
banking can develop efficient and accurate measurement tools
(Batterham et al., 2015). Batterham et al. have developed an OCD item
bank for use with adult populations, and members of that team are in
the process of creating one for use with 7–18 year olds. Importantly,
item banking provides an option to equate different youth OCD scales
that measure the same broad construct but differ in item content and
symptom severity, employing IRT analysis to generate scores on a
common interpretable metric. Additionally, youth item banks can be
11
Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12
equated with adult item banks so that psychopathology can be mea-
sured over the life course, and, as part of a suite of instruments cali-
brated against underlying latent structures, item banks can account for
comorbidity better than traditional assessments can (Batterham et al.,
2015).
Acknowledgements
Thanks to Eiko Fried for providing additional information about his
content overlap analysis methods.
Role of Funding Sources
This work was supported by a UNSW Medicine, Neuroscience,
Mental Health and Addiction Theme and SPHERE Mindgardens CAG
collaborative research seed funding grant (PS45925). UNSW had no
role in the study design, collection, analysis or interpretation of the
data, writing the manuscript, or the decision to submit the paper for
publication.
Contributors
Authors MS and RV designed the study. Author RV extracted the
items from scales and conducted the statistical analysis. Authors RV,
MS, TS and JG wrote the manuscript, and all authors contributed to and
have approved the final manuscript.
Conflict of Interest
All authors declare that they have no conflicts of interest.
Appendix A. Supplementary material
Supplementary data associated with this article can be found in the
online version at doi:10.1016/j.jocrd.2018.10.005.
References
American Psychiatric Association (2013). Diagnostic and statistical manual of mental dis-
orders (5th ed.). Washington, DC: American Psychiatric Publishing.
Batterham, P. J., Brewer, J. L., Tjhin, A., Sunderland, M., Carragher, N., & Calear, A. L.
(2015). Systematic item selection process applied to developing item pools for as-
sessing multiple mental health problems. Journal of Clinical Epidemiology, 68(8),
913–919. https://doi.org/10.1016/j.jclinepi.2015.03.022.
Berg, C. J., Rapoport, J. L., & Flament, M. (1986). The leyton obsessional inventory-child
version. Journal of the American Academy of Child Psychiatry, 25(1), 84–91. https://
doi.org/10.1016/S0002-7138(09)60602-6.
Berman, N. C., & Abramowitz, J. S. (2010). Recent developments in the assessment and
treatment of pediatric obsessive-compulsive disorder. Child & Youth Care Forum,
39(2), 125–127. https://doi.org/10.1007/s10566-010-9096-4.
Douglass, H. M., Moffitt, T. E., Dar, R., McGee, R. O. B., & Silva, P. (1995). Obsessive-
compulsive disorder in a birth cohort of 18-year-olds: Prevalence and predictors.
Journal of the American Academy of Child & Adolescent Psychiatry, 34(11), 1424–1431.
https://doi.org/10.1097/00004583-199511000-00008.
Foa, E. B., Coles, M., Huppert, J. D., Pasupuleti, R. V., Franklin, M. E., & March, J. (2010).
Development and validation of a child version of the obsessive compulsive inventory.
Behavior Therapy, 41(1), 121–132. https://doi.org/10.1016/j.beth.2009.02.001.
Freeman, J., Flessner, C. A., & Garcia, A. (2011). The children's yale-brown obsessive
compulsive scale: Reliability and validity for use among 5 to 8 year olds with ob-
sessive-compulsive disorder. Journal of Abnormal Child Psychology, 39(6), 877–883.
https://doi.org/10.1007/s10802-011-9494-6.
Fried, E. I. (2017). The 52 symptoms of major depression: Lack of content overlap among
seven common depression scales. Journal of Affective Disorders, 208, 191–197.
https://doi.org/10.1016/j.jad.2016.10.019.
Heyman, I., Fombonne, E., Simmons, H., Ford, T., Meltzer, H., & Goodman, R. (2003).
Prevalence of obsessive-compulsive disorder in the British nationwide survey of child
mental health. International Review of Psychiatry, 15(1–2), 178–184. https://doi.org/
10.1080/0954026021000046146.
Iniesta-Sepúlveda, M., Rosa-Alcázar, A. I., Rosa-Alcázar, Á., & Storch, E. A. (2014).
Evidence-based assessment in children and adolescents with obsessive–compulsive
disorder. Journal of Child and Family Studies, 23(8), 1455–1470. https://doi.org/10.
1007/s10826-013-9801-7.
Lewin, A. B., Park, J. M., & Storch, E. A. (2013). Obsessive-compulsive disorder in chil-
dren and adolescents. In R. A. Vasa, & A. K. Roy (Eds.). Pediatric anxiety disorders: A
R. Visontay et al.
clinical guide (pp. 157–175). New York, NY: Springer New York.
Lewin, A. B., & Piacentini, J. (2010). Evidence-based assessment of child obsessive
compulsive disorder: Recommendations for clinical practice and treatment research.
Child & Youth Care Forum, 39(2), 73–89. https://doi.org/10.1007/s10566-009-
9092-8.
Peris, T. S., Bergman, R. L., Langley, A., Chang, S., McCracken, J. T., & Piacentini, J.
(2008). Correlates of accommodation of pediatric obsessive-compulsive disorder:
Parent, Child, and family characteristics. Journal of the American Academy of Child &
Adolescent Psychiatry, 47(10), 1173–1181. https://doi.org/10.1097/CHI.
0b013e3181825a91.
Piacentini, J., Bergman, R. L., Keller, M., & McCracken, J. (2003). Functional impairment
in children and adolescents with obsessive-compulsive disorder. Journal of Child and
Adolescent Psychopharmacology, 13(Suppl. 1), S61–S69. https://doi.org/10.1089/
104454603322126359.
Piqueras, J. A., Rodríguez-Jiménez, T., Ortiz, A. G., Moreno, E., Lázaro, L., & Godoy, A.
(2015). Validation of the short obsessive–compulsive disorder screener (SOCS) in
children and adolescents. BJPsych Open, 1(1), 21–26. https://doi.org/10.1192/bjpo.
bp.115.000695.
Rapoport, J. L., Inoff-Germain, G., Weissman, M. M., Greenwald, S., Narrow, W. E.,
Jensen, P. S., et al. (2000). Childhood obsessive–compulsive disorder in the NIMH
MECA study: Parent versus child identification of cases. Journal of Anxiety Disorders,
14(6), 535–548. https://doi.org/10.1016/S0887-6185(00)00048-7.
Selles, R. R., Storch, E. A., & Lewin, A. B. (2014). Variations in symptom prevalence and
clinical correlates in younger versus older youth with obsessive–compulsive disorder.
Child Psychiatry & Human Development, 45(6), 666–674. https://doi.org/10.1007/
s10578-014-0435-9.
Sookman, D., Abramowitz, J. S., Calamari, J. E., Wilhelm, S., & McKay, D. (2005).
Subtypes of obsessive-compulsive disorder: Implications for specialized cognitive
behavior therapy. Behavior Therapy, 36(4), 393–400. https://doi.org/10.1016/
S0005-7894(05)80121-2.
Stewart, S. E., Hu, Y.-P., Hezel, D. M., Proujansky, R., Lamstein, A., Walsh, C., et al.
(2011). Development and psychometric properties of the OCD family functioning
(OFF) Scale. Journal of Family Psychology, 25(3), 434–443. https://doi.org/10.1037/
a0023735.
Storch, E. A., Björgvinsson, T., Riemann, B., Lewin, A. B., Morales, M. J., & Murphy, T. K.
12
Journal of Obsessive-Compulsive and Related Disorders 21 (2019) 6–12
(2010). Factors associated with poor response in cognitive-behavioral therapy for
pediatric obsessive-compulsive disorder. Bulletin of the Menninger Clinic, 74(2),
167–185. https://doi.org/10.1521/bumc.2010.74.2.167.
Storch, E. A., Khanna, M., Merlo, L. J., Loew, B. A., Franklin, M., Reid, J. M., et al. (2009).
Children's Florida obsessive compulsive inventory: Psychometric properties and
feasibility of a self-report measure of obsessive–compulsive symptoms in youth. Child
Psychiatry and Human Development, 40(3), 467–483. https://doi.org/10.1007/
s10578-009-0138-9.
Storch, E. A., Merlo, L. J., Larson, M. J., Bloss, C. S., Geffken, G. R., Jacob, M. L., et al.
(2008). Symptom dimensions and cognitive-behavioural therapy outcome for pe-
diatric obsessive-compulsive disorder. Acta Psychiatrica Scandinavica, 117(1), 67–75.
https://doi.org/10.1111/j.1600-0447.2007.01113.x.
Storch, E. A., Merlo, L. J., Larson, M. J., Geffken, G. R., Lehmkuhl, H. D., Jacob, M. L.,
et al. (2008). Impact of comorbidity on cognitive-behavioral therapy response in
pediatric obsessive-compulsive disorder. Journal of the American Academy of Child &
Adolescent Psychiatry, 47(5), 583–592. https://doi.org/10.1097/CHI.
0b013e31816774b1.
Storch, E. A., Murphy, T. K., Adkins, J. W., Lewin, A. B., Geffken, G. R., Johns, N. B., et al.
(2006). The children's Yale-Brown obsessive–compulsive scale: Psychometric prop-
erties of child- and parent-report formats. Journal of Anxiety Disorders, 20(8),
1055–1070. https://doi.org/10.1016/j.janxdis.2006.01.006.
Uher, R., Heyman, I., Mortimore, C., Frampton, I., & Goodman, R. (2007). Screening
young people for obsessive-compulsive disorder. British Journal of Psychiatry, 191(4),
353–354. https://doi.org/10.1192/bjp.bp.106.034967.
Uher, R., Heyman, I., Turner, C. M., & Shafran, R. (2008). Self-, parent-report and in-
terview measures of obsessive–compulsive disorder in children and adolescents.
Journal of Anxiety Disorders, 22(6), 979–990. https://doi.org/10.1016/j.janxdis.2007.
10.001.
Watson, H. J., & Rees, C. S. (2008). Meta-analysis of randomized, controlled treatment
trials for pediatric obsessive-compulsive disorder. Journal of Child Psychology and
Psychiatry, 49(5), 489–498. https://doi.org/10.1111/j.1469-7610.2007.01875.x.
Williams, M. T., Mugno, B., Franklin, M., & Faber, S. (2013). Symptom dimensions in
obsessive-compulsive disorder: Phenomenology and treatment outcomes with ex-
posure and ritual prevention. Psychopathology, 46(6), 365–376. https://doi.org/10.
1159/000348582.