Professional Documents
Culture Documents
Clarke Et Al 2016 The Relationship Between Negative Problem Orientation and Worry A Meta Analytic Review
Clarke Et Al 2016 The Relationship Between Negative Problem Orientation and Worry A Meta Analytic Review
James Bowness Clarkea, b, Michelle Ford b, Sara Hearyb, Jacqui Rodgersa, Mark H.
Freestona, b
a
Newcastle University
b
Northumberland, Tyne and Wear NHS Trust
Abstract
Generalised Anxiety Disorder (GAD) can have a significant detrimental effect on the lives of sufferers. Research into
the psychological process involved in worry has an important role in the prevention and treatment of problems such
as GAD. Negative problem orientation (NPO) has been identified as an important psychological construct involved
in worry (Gosselin, Ladouceur & Pelletier, 2005). The focus of this meta-analytic review was to further our
understanding of the relationship between NPO and worry, and a related construct called intolerance of uncertainty
(IU). The review identified 31 articles reporting correlations between measures of NPO and worry. These articles
detailed a total of 33 independent effect sizes. Using a random-effects model and raw and attenuation-corrected
values, NPO was found to be significantly correlated to worry (r = 0.57 , CI = 0.55-0.60, k = 33, N = 5376). The type
of NPO measure moderated the relationship between NPO and worry. In addition, NPO was found to be significantly
correlated with IU (r= 0.71, CI = 0.69-0.73, k = 14, N = 3072) and more strongly correlated with IU than with worry
(p<.05). The findings have implications for theory, research and practice. Issues in the measurement of NPO are
discussed, and the relationship between NPO and IU is considered.
© Copyright 2017 Textrum Ltd. All rights reserved.
Keywords: Negative Problem Orientation, Worry, Generalized Anxiety Disorder, GAD, Meta-Analysis
Correspondence to: Mark.Freeston@newcastle.ac.uk
Received 11-Oct-2013; received in revised form 05-Jan-2015; accepted 12-Dec-2014
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 320
Table of Contents
Introduction
Generalised Anxiety Disorder
The Intolerance of Uncertainty Model
Negative Problem Orientation
Review objectives
Method
Literature Search
Search terms.
Search strategy.
Inclusion/exclusion criteria.
Selection of studies.
Meta-Analytic Method
Inter-Rater Reliability.
Independent Effect Sizes.
Outliers.
Transformation of r.
Homogeneity.
Publication Bias.
Correction for Attenuation.
Effect Size Coding.
Sources of Heterogeneity.
Moderator Analysis.
Results
Relationship between Negative Problem Orientation and Worry
Consideration of the Exclusion of Outliers.
Moderator Analysis.
Relationship between Negative Problem Orientation and Intolerance of Uncertainty.
Moderator Analysis.
Relationship between Negative Problem Orientation and Intolerance of Uncertainty and Worry
Discussion
Negative Problem Orientation definition and measurement
Relationship between Negative Problem Orientation and Intolerance of Uncertainty
Limitations
Future Directions
Conclusion
References
Appendices
Introduction
internal experiences, and propose that individuals with GAD attempt to variously avoid: emotion-laden stimuli, such
as vivid images (The Avoidance Model; Borkovec, Alcaine & Behar, 2004); uncertainty (The IU Model; Dugas,
Gagnon, Ladouceur & Freeston, 1998); emotions (The Emotion Dysregulation Model; Mennin, Heimberg, Turk &
Fresco, 2005); internal experiences (The Acceptance Based Model; Roemer & Orsillo, 2002); or distressing meta-
cognitions about worry (Wells, 1995).
The models have conceptual differences and can be classified into three categories: cognitive models, namely the
Intolerance of Uncertainty (IU) and Meta-Cognitive Model; emotional/behavioural models, namely the Emotion
Dysregulation and Acceptance-Based Model; and integrative models, namely the Avoidance Model. The conceptual
differences have implications for treatments that follow them. For example, treatments that follow the cognitive
models focus on cognitive restructuring through techniques such as thought challenging and behavioural
experiments, and in the case of the IUM model, using simple problem solving with reality-based worries to address
negative problem orientation and intolerance of uncertainty. In contrast, treatments following the emotion and
behavioural models focus on increasing emotional tolerance through techniques such as emotional skills training or
emotional literacy. To further increase treatment efficacy, the key psychological constructs involved in GAD need to
be established.
solving situations, labelled as negative problem orientation (NPO). As a result, they may avoid engaging in problem
solving or engage in a way that does not lead to clear resolution, which subsequently maintains their worry. While
NPO is a key facet of the IU model, it does not feature in the other GAD models reviewed by Behar et al. (2009). It
is therefore timely to review the NPO construct in detail and its relationship with worry.
Q; Gosselin, Ladouceur & Pelletier, 2005) tried to rectify this by conceptualising NPO as a purely cognitive
phenomenon. Items on the NPO-Q include: ‘I often have the impression that my problems cannot be solved’, ‘I often
see my problems as bigger than they really are’ and ‘I consider problems to be obstacles that interfere with my
functioning’.
As NPO has moved towards being measured as a purely cognitive psychological process, it raises the question
whether individuals do in fact have a general negative predisposition towards problem solving situations, or whether
they have an aversion to the uncertainty evoked by problem solving scenarios, as captured by the Intolerance of
Uncertainty Scale (IUS, Freeston, Rhéaume, Letarte, Dugas, Ladouceur, 1994).
Review objectives
Within the current literature a number of studies have examined the relationship between NPO and worry.
Nevertheless, given the variability regarding the definition and measurement of NPO the picture remains unclear. It
is therefore timely to review the relationship between NPO and worry, and the impact that the type of NPO measure
has on this relationship.
A secondary aim of the research is to examine the relationship between NPO and IU. Recent developments in the
conceptual understanding of NPO and IU, have led to the NPO-Q (Gosselin et al., 2005) and the IUS (Birrell, Meares,
Wilkinson, & Freeston, 2011) containing similar items (Appendix 1). There is strongest overlap with the factor of the
IUS-12 that has been variously labelled as inhibitory IU (Carleton, 2012) or behavioural paralysis (Birrell et al., 2011).
As a result, it is unclear whether NPO would be best understood as an alternative measure of IU, which makes it
redundant, or an expression of IU, which adds to our understanding of IU and GAD.
Some research has attempted to explore whether NPO is distinct from IU by examining whether NPO still has a
relationship with worry when controlling for IU (Carleton, Gosselin & Asmundson, 2010). Nevertheless there are not
enough studies in this area to draw any firm conclusions. Another way of examining the relationship between NPO
and IU, is to establish the degree of shared variance between the constructs. If the shared variance between IU and
NPO is higher than the relationship between NPO and worry (or IU and worry), this could suggest that NPO and IU
are assessing the same construct, while lower levels may indicate that the constructs have unique features.
While the review aims can be addressed by a number of methodologies, the use of a meta-analysis has a number
of advantages. Firstly, it provides a comprehensive review of the relationship between NPO and worry by including
all available data in the field. Secondly, through the use of correction for attenuation procedures the variance
attributable to measurement error can be factored out of the analysis. And finally, through the partitioning of variance,
the impact of the type of NPO measure as a potential moderator can be examined, helping establish whether NPO
appears to be more accurately conceptualised as a cognitive-emotional, or purely cognitive, variable. While a recent
meta-analysis examined the relationship between IU and worry (Gentes & Ruscio, 2011), this has not been
conducted for NPO and worry, nor NPO and IU.
The review objectives are to:
1. Use all currently available data to establish the mean correlation between NPO and worry.
2. Identify whether the type of NPO measure moderates the relationship between NPO and worry.
3. Use all currently available data to establish the mean correlation between NPO and IU.
Method
Literature Search
Search terms.
Search terms were selected by identifying key words used within the field. Consideration was given to searching for
NPO. However, due to NPO being a relatively new construct, the search strategy was broadened to capture the
whole problem solving literature: the search terms ‘Problem Solving’ and ‘Problem Orientation’ were used. The
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 324
search term ‘Worry’ was used. Due to worry being a central feature of GAD, ‘GAD’ and ‘Generalized Anxiety Disorder’
were also used as search terms. Wild cards (*) were used to ensure that all variations of the search terms were
captured
Negative Problem Orientation: Problem Solv* (solving, solving strategies, solving approaches) OR Problem
Orientation* (Orientation, Orientations)
Worry: Worr* (worry, worrying, worried) OR Generali?ed (Generalised, Generalized) Anxiety Disorder* (Disorder,
Disorders, Disordered) OR GAD
Search strategy.
In addition to database searches, a range of additional strategies were employed. Four prominent authors in the field
(Davey, Dugas, Ladouceur & Ruscio) were contacted via email and asked if they knew of any published or
unpublished data that could contribute to the meta-analysis. Davey was selected due to him being one of the first
authors to look at the relationship between problem-solving, NPO and worry; Dugas and Ladouceur, due to them
being prolific researchers in the field; and Ruscio due to her conducting a meta-analysis in the worry and IU field.
Two additional articles were identified, one of which met inclusion criteria. A number of relevant journals were
searched by hand (Behaviour Research and Therapy, Cognitive Therapy and Research, Journal of Anxiety Disorders
& Behaviour and Cognitive Psychotherapy), no additional literature was identified. The Journal of Anxiety Disorders
was selected due to it being relevant to GAD. The Journal of Behaviour Research and Therapy, Cognitive Therapy
and Research, and Behaviour and Cognitive Psychotherapy were all selected due to them being relevant to cognitive
behaviour therapy (and therefore the IU model of GAD). To increase the chance of capturing grey literature, the
search engines EThOS, WorldCat and ProQuest Dissertation Publishing were searched. No additional literature was
identified. Dugas was identified as a key author in the field. An author search, conducted in Web of Knowledge,
Embase and Psycinfo, produced 55 hits: 17 were identified as duplicates and the remaining 38 articles did not meet
inclusion criteria. The reference list from a review of problem solving was examined (Nezu, 2004). NPO is a relatively
new construct and within the problem solving literature a number of earlier studies have examined similar phenomena
such as problem solving confidence. No additional articles were identified. A citation search of articles citing a key
paper in the area (Dugas et al., 1998) was conducted. The paper had subsequently been cited 172 times. Thirty four
of these articles were duplicates, while the remaining papers (138) did not meet inclusion criteria.
Inclusion/exclusion criteria.
Correlational research that reported a zero order r value between a measure of worry and NPO was included. As
worry is a commonly occurring process present in non-clinical populations (Dupuy & Beaudoin, 2001), research
conducted in both analogue and clinical populations was included. Worry and anxiety have been identified as
separate constructs (Davey et al., 1992). Studies that used anxiety, rather than worry, as a primary outcome measure
were therefore excluded from the review; the State-Trait Anxiety Inventory (Spielberger, 2010) and the Hospital
Anxiety and Depression Scale (Zigmond & Snaith, 1983) were not considered suitable.
Between-groups research that examined NPO in a low versus high worry group was included. Studies were required
to include groups that differed in worry levels, and to report mean NPO scores for each group. GAD is characterised
by chronic and persuasive worry (APA, 1994) and therefore GAD patients were deemed to represent a high worry
group. Between-groups research using other clinical disorders was not included, due to the worry being specific to
one domain (e.g. health anxiety) or acute episodes of worry specific to the main problem (e.g. panic disorder), rather
than persistent general themes.
No papers were initially excluded based on their language of publication. However, papers that could not be
adequately translated despite the help of interpreters (e.g. Japanese) were not included. Both published and
unpublished research, including a conference presentation and a doctorate thesis, was included in the review.
Theoretical and conceptual reviews were excluded. No studies were excluded based on their date of publication, the
earliest publication dates varied according to the database used. No studies were excluded on the basis of age of
participants. The inclusion criteria aimed to increase the coverage of the review.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 325
Selection of studies.
A total of 1552 papers were identified through the literature search. Six hundred and fifty seven were identified as
duplicates and removed, leaving a total of 895. Identified papers were reviewed at the level of title and abstract
against the inclusion/exclusion criteria; 852 papers did not meet inclusion criteria and were removed, leaving a total
of 43 papers. Two additional papers in the process of being published were identified through the research team
(Smith, 2012; Bottesi, Heary, Ham, Peden & Freeston, 2012). Two papers were identified through author contact
(Meeten, Dash, Scarlet & Davey, 2012; Ruscio & Sietchik, 2007). A full text review of the 47 studies was conducted
to determine whether the studies reported the necessary statistics, used an appropriate measure of NPO and worry,
and could be adequately translated. This text review revealed that eight papers did not report the appropriate statistics
(Carleton et al., 2010; Drews & Hazlett-Stevens, 2007; Dugas et al., 1998; Dugas et al., 1995; Dupuy & Ladouceur,
2008; Kertz & Woodruff-Borden, 2013; Laugesen, Dugas & Bukowski, 2003; Meeten et al., 2012), four papers did
not include an adequate measure of NPO (Szabo & Lovibond, 2002, 2004 & 2006; Thompson, Webber &
Montgomery, 2002), two did not use an adequate worry measure (Anderson & Goddard, 2009; Robichaud & Dugas,
2005) and two could not be adequately translated (Sugiura, 1999, 2002). This left a total of 31 papers. One paper
used a within-group design to examine the relationship between NPO and worry in a sample of chronic pain patients
(De Vlieger et al., 2006). The worry and NPO measures used were not pain specific. While the relationship between
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 326
NPO and worry may have been different within certain clinical populations, including this study increased the
coverage of the study. No other papers conducted in specific clinical populations met inclusion criteria and were
therefore not included. The paper selection process is outlined below in Figure 2.
Meta-Analytic Method
The overall meta-analytic method was modelled on Gentes and Ruscio (2011) which uses a standard and thorough
approach (based on Borenstein, Hedher, Higgins & Rothstein, 2009) and addressed a similar question to the present
review (the relationship between a psychological construct, intolerance of uncertainty, and worry or GAD). All primary
sources for formulae were obtained and are referenced at the appropriate point in the following paragraphs. Formulae
were input into spreadsheets, and data from each of the primary sources articles were tested to ensure accuracy.
The formulae will not be repeated here as they are reported in detail by Gentes and Ruscio (2011). Excel was used
for data extraction and synthesis using spreadsheets developed by the last author (MHF).
Inter-Rater Reliability.
To check for inter-rater reliability, relevant data were independently coded into separate spreadsheets by the first
(JBC) and third (SH) authors of the study. There was a 94% agreement rate; discrepancies were primarily identified
in papers that were not written in English. All differences were reviewed by the author and second researcher with
reference to original papers. This procedure led to a 100% agreement on all effect sizes and study characteristics
detailed in the study.
Outliers.
The sample-adjusted meta-analytic deviance (SAMD) statistic (Hutcutt & Arthur, 1995) was calculated to test for the
presence of statistical outliers. The distribution of SAMD statistics approximates a normal t distribution (Hutcutt &
Arthur, 1995). It was important to note that elimination of outliers may lead to the underestimation of true variability,
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 327
while failure to eliminate outliers may lead to overestimation of variability (Huffcutt & Arthur, 1995). Examination of
the distribution of the coefficients highlighted candidates for Winsorization.
Transformation of r.
The weighted mean effect size of each study was calculated. Due to correlation coefficients having a skewed
standard error formulation (Rosental, 1991), Fisher’s Zr transform was used to transform effect sizes. Effect sizes
were then weighted by the inverse of the squared standard error of the Z-transformed correlation.
Homogeneity.
The homogeneity of the data was analysed using the Q statistic. The Q statistic is distributed as a Chi square with k
– 1 degrees of freedom (k is the number of effect sizes; Hedges & Olkin, 1985). The heterogeneity of the data was
examined using the I2 statistic (Higgins, Thompson, Deeks & Altman, 2003). Benchmarks for low medium and high
heterogeneity, of 25%, 50% and 75% respectively are outlined by Higgins et al. (2003).
Publication Bias.
Referring to the phenomena of papers with significant findings having a higher chance of publication, publication bias
can lead to the over reporting of effect sizes in the literature. Orwin’s fail safe N (FSN), indicates the minimum number
of unpublished findings with an effect size of no interest (r = .10) that would need publication to reduce the mean
observed effect size to zero. Funnel plots were generated but due to the small number of studies they were not very
informative.
Sources of Heterogeneity.
Two statistical models are used for meta-analysis, the fixed and random-effects models. The fixed-effects model
assumes that there is one true effect size which underlies all studies in the analysis and any observed differences
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 328
are attributable to sampling error; findings are typically generalizable to studies included in the review. The random-
effects model posits that true effect size may vary from study to study; findings are generalizable beyond studies
included in the review (Borenstein et al., 2009). Due to the review attempting to synthesize research that measured
and conceptualised NPO in different ways, a random effects model was chosen.
Moderator Analysis.
To examine the type of NPO measure as a possible moderator, papers were divided into subsets that had used
theoretically similar NPO measures. The three subsets were initially identified by the first author of the paper and
then reviewed by the last author. It was agreed that these groups represented theoretical shifts in how NPO has been
conceptualised within the literature. The first subgroup, included studies that measured NPO using a measure of
problem solving confidence (e.g. the PSI; Heppner & Petersen, 1982); the second, studies that captured NPO using
a version of the SPSI (D’Zurilla & Nezu, 1990) or the SPSI-R (Maydeu-Olivares & D’Zurilla, 1996), which both
conceptualise NPO as a cognitive, emotional and behavioural construct; and the third, studies that conceptualised
NPO as a cognitive construct (The NPO-Q; Gosselin et al., 2005). Both the first author and a second researcher
independently assigned the papers to the same groups, indicating that the process was reliable.
The review considered the impact of additional moderators, which were participant age and gender.
Age was considered as a possible moderator. While the majority of NPO research has been conducted in adult
samples, more recent research has begun to examine the role of NPO in worry, within child and adolescent samples
(e.g. Parkinson & Creswell, 2011; Wilson & Hughes, 2011). These studies have reported high correlations between
NPO and worry.
Gender was also considered as a moderator. Women have been found to be more likely to report higher levels of
worry (Lewinsohn, Gotlib, Lewinsohn, Seeley & Allen, 1998) and NPO (D’Zurilla, Maydeu-Olivares & Kant, 1998)
compared to men. Thus, while men may under-report their levels of NPO and worry, women are perhaps more likely
to accurately report if they are experiencing high levels of NPO and worry. Differential reporting by men and women
and any associated restricted range effects could influence the strength of correlations, so the percentage of women
in the sample was considered as a moderator.
Results
control participants; GAD = Generalized Anxiety Disorder; PD = Panic Disorder; PSI = Personal Solving Inventory; PSRM-C =
Problem Solving Related Meta-Cognitive Questionnaire; SPSI = Social Problem Solving Inventory; SPSI-R = Social Problem
Solving Inventory Revised; NPOQ = Negative Problem Orientation Questionnaire; PSWQ = Penn State Worry Questionnaire;
PSWQ-C = Penn State Worry Questionnaire-Child Version; WAQ = Worry and anxiety questionnaire; WDQ = Worry Domains
Questionnaire; SWS = Student Worry Scale.
All studies 5376 33 0.57 0.55-0.60 94.07** 65.98 157 0.65 0.62-0.67
*= Significant at the level of 0.05; ** = Significant at the level of 0.01; N = Number of participants; k = Number of papers; CI =
Confidence Intervals; FSN =
Orwin’s fail safe N. Note included studies assigned half weights resulting in some N values including decimal points.
The overall mean r between NPO and worry was 0.57 (95% IC = .55-.60) (Table 2). Orwin’s fail safe N indicated that
an additional 157 studies with an effect size of no interest would need to be published in order to reduce the mean
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 330
correlation to zero. As shown in Table 2, Q values revealed that the effect size distributions were significantly
heterogeneous (p<0.01). Consideration was given to removing outliers in the dataset.
Moderator Analysis.
Moderators within the data were examined by partitioning the heterogeneity of the effect sizes. Studies were
separated into the subgroups previously described. This allowed the within and between group heterogeneity of each
group of effect sizes to be examined (Lipsey & Wilson, 2001). Heterogeneity was then partitioned by separating the
total homogeneity statistic (Qtotal), into the proportion explained by the categorical between groups variable (Qb), and
the residual pooled within groups portion (Qw) (computed by calculating and then summing Q for each group of effect
sizes). The following formula was used:
Qb = Qtotal - Qw
The Q is distributed as chi square. The degrees of freedom for Qb were calculated by the number of groups (j) – 1.
The degrees of freedom for Qw were calculated by k-j, were k equals the number of effect sizes. The portioned
variance is outlined in Table 3 below.
Table 3: Moderation of the Relationship between NPO and Worry, by NPO Measure
Q QPSC QPO QNPOQ Qw df (Qw) p (Qw) I2 (Qw) Qb df (Qb) p (Qb) I2% (Qb)
Random 94.07 22.19 16.77 23.21 62.17 11 0.00 82.31 31.90 2 0.00 93.73
Effects
Model
PSC = Problem solving confidence measures; PO = Problem orientation measures; NPOQ = Negative Problem Orientation
Questionnaire; df = Degrees of freedom; Qw = Q statistic within groups; Qb = Q statistic between groups
As shown in Table 3, a large Q value indicated that there was variability between studies above that attributable to
measurement error. Measure was a significant moderator (Qb). Within group heterogeneity (Qw) was identified,
indicating that even when partitioned into measure specific subgroups, the studies still demonstrated significant within
subgroup variability.
The mean correlation between NPO and worry was established for each subgroup of studies. A mean correlation of
r = 0.47 (95% CI = .41-.52) was identified for studies that used a measure of problem solving confidence; r = 0.61
(95% CI = 0.57-0.65) for studies that used a measure of problem solving orientation; and r = 0.59 (0.56-0.61) for
studies that used the Negative Problem Orientation Questionnaire (NPOQ; Gosselin et al., 2005).
The confidence intervals between the subgroups of studies using a measure of problem orientation and the NPOQ
overlapped. However, there was no overlap between studies using a measure of problem solving confidence and
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 331
other measures. It could therefore be asserted, with a 95% degree of confidence, that the correlations between these
groups were different. In summary, studies using a measure of problem solving confidence had a weaker correlation
with worry compared to studies using a measure of problem orientation or the NPOQ.
The influence of additional moderators in the data was considered. Average participant age and the percentage of
female participants, were correlated with the study NPO-worry correlation. For the mean age of participants, r = -0.33
(-0.62 - -0.01) indicating that as participant age increased, the strength of the relationship between NPO and worry
decreased. For percentage of female participants r = 0.16 (-0.18 – 0.48), but the CIs includes 0, indicating that the
relationship was not significant.
Table 4: Summary of Studies Included in the Meta-Analysis Examining the Relationship between NPO and IU
Sample Mean SE NPO IU
Author Type N Age r Measure Measure
SPSI-
Dugas (1997) Analogue 285 23.50 0.65 0.06 abridged IUS
SPSI-
Dugas (2005) GAD vs PD 45 38.62 0.40 0.15 abridged IUS
Dugas (2007) ab GAD 84 37.76 0.61 0.11 SPS-R-Short IUS
Gosselin (2002) Analogue 346 22.60 0.69 0.05 SPSI-R IUS
Gosselin (2005) b Analogue 172* 21.72 0.67 0.08 SPSI-R IUS
Robichaud (2003) Analogue 217 22.35 0.65 0.07 SPS-R-Short IUS
Barahmand (2008) b Analogue 197 17.49 0.66 0.07 NPOQ IUS
Bottesi, Heary, Ham, Peden & Freeston,
2012) Analogue 101 19.39 0.64 0.10 NPOQ IUS
Fergus (2010) Analogue 414 19.00 0.82 0.05 NPOQ IUS
Fergus (2011) Study 1 Analogue 362 19.50 0.70 0.05 NPOQ IUS
Fergus (2011) Study 2 Analogue 363 19.50 0.71 0.05 NPOQ IUS
Gosselin (2005) Analogue 172* 21.72 0.73 0.08 NPOQ IU
Rausch (2011) Analogue 102 16.00 0.74 0.10 NPOQ IUS
Smith (2012) b Analogue 212 12.85 0.79 0.07 NPOQ-C IUSC
a= b
Removed as an outlier in first stage; = Removed as a outlier in second stage; * = Assigned half weights; Analogue = healthy
control participants; GAD = Generalized Anxiety Disorder; PD = Panic Disorder; PSI = Personal Solving Inventory; PSRM-C =
Problem Solving Related Meta-Cognitive Questionnaire; SPSI = Social Problem Solving Inventory; SPSI-R = Social Problem
Solving Inventory Revised; NPOQ = Negative Problem Orientation Questionnaire; IUS = Intolerance of Uncertainty Scale; IU=
Intolerance of Uncertainty questionnaire; IUSC = Intolerance of Uncertainty Scale for Children.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 332
N = Number of participants; k = Number of papers; CI = Confidence Intervals; FSN = Orwin’s fail safe N.
As shown by Table 5, both subgroups were found to be homogeneous. While the Q value for all studies substantially
decreased, it was still significant; the influence of the type of NPO measure as a moderator was examined.
Moderator Analysis.
As highlighted by Table 6, a large Q value indicated that there was variability between studies above that attributable
to measurement error
The moderator effect (QB) was significant, indicating that measure was a significant moderator of the relationship
between NPO and IU. Within group heterogeneity (Qw) was not significant, indicating that when controlling for the
type of NPO measure as a moderator the studies were homogeneous.
Reported correlation between negative problem orientation and intolerance of uncertainty. The overall mean r
between NPO and IU was reported to be 0.71 (0.69-0.73) (Table 4). For studies that used a measure of problem
orientation the average correlation with IU was 0.66 (0.63-0.69), and for studies using the NPOQ, r = 0.74 (0.71-
0.76). There was no overlap between the 95% confidence intervals for studies using a measure of problem orientation
and studies using the NPOQ. Thus, the correlations between NPO and IU for these subgroups were different.
Nevertheless, it is worth noting that the power of this finding is relatively low with a relatively small k and relatively
wide confidence intervals.
Table 6: Moderation of the Relationship between NPO and IU, by NPO Measure
Q QPO QNPOQ Qw df (Qw) p (Qw) I2 (Qw) Qb df (Qb) p (Qb) I2% (Qb)
Random 35.44 2.23 21.59 23.82 12 0.02 49.62 11.62 2 0.00 82.79
Effects
Model
PO = Problem orientation measures; NPOQ = Negative Problem Orientation Questionnaire; df = Degrees of freedom; Qw = Q
statistic within groups; Qb = Q statistic between groups
Discussion
The primary aim of the review was to use all currently available data to examine the role of negative problem
orientation (NPO) in worry. The review synthesised research that had conceptualised NPO from somewhat different
perspectives. The identified literature was found to be significantly heterogeneous and a random-effects model was
deemed the most appropriate method to analyse the data. An overall mean correlation of 0.57 (0.55-0.60) between
NPO and worry was identified, when corrected for attenuation, rho = 0.65 (0.62-0.67). In summary, the review found
that NPO is highly correlated with worry.
A second aim of the research was to examine factors that may moderate the relationship between NPO and worry.
Through partitioning variance, the type of NPO measure was identified as a significant moderator. Earlier research
that attempted to assess NPO by measuring problem solving confidence was moderately correlated to worry (r =
0.47, 0.41-0.52), while later research that conceptualised NPO as a stand-alone construct was strongly correlated to
worry (r = 0.59, 0.56 - 0.61). Within the recent literature, no difference was identified between measures that
conceptualised NPO as a cognitive-emotional (r = 0.61, 0.57 - 0.65), compared to a purely cognitive (r = 0.59, 0.56-
0.61) construct. A regression analysis indicated that participant age and gender should be considered as potential
moderators in future research, despite neither of them accounting for significant unique variance in the present
review.
A third aim of the review was to examine the relationship between NPO and intolerance of uncertainty (IU), defined
as ‘a fear of the unknown’ (Carleton, 2012). It is possible that rather than measuring negative predispositions to
problem solving scenarios per se, NPO measures may instead measure aversion to the uncertainty created by
problem solving scenarios (IU). The review found that these NPO and IU were highly correlated (r = 0.71, 0.69-0.73),
when corrections were made for attenuation, rho = 0.78 (0.76-0.80). NPO had a stronger correlation with IU than with
worry. Measure did not moderate these relationships.
Relationship between Negative Problem Orientation and Worry
The present review found that NPO is strongly correlated to worry. This is consistent with the IU model (Dugas et al.,
1998), which proposes that NPO plays an important role in the maintenance of worry. In order to understand the role
that NPO plays in worry it is worthwhile examining the IU model in more detail. The first conceptualisation of the IU
model (Dugas et al., 1998) proposes that one of the outcomes of worry was NPO, which in turn maintains worry. This
indicates that NPO is activated by worry and subsequently plays a key role in its maintenance. Recent revisions of
the IU model propose that the relationship between IU and worry is mediated by NPO, positive beliefs about worry
(PBW), cognitive avoidance (CA; Bottesi, Heary, Ham, Peden & Freeston, 2012) and a negative interpretation bias
(NIB; Koerner, Dugas, Gosselin & Langlois, 2013). In the latter revision of the IU model, these mediators are proposed
to have a sequential relationship, with IU initially activating NPO and PBW, which in turn prompts CA and NIB
(Koerner et al., 2013). This development proposes a subtly different role for NPO by proposing that it is initially
activated by IU, and then in turn activates other mediators involved in the worry process. While it was beyond the
scope of the present review, and the currently available published data, to clarify the role that NPO plays in worry,
these developments highlight an area of interest for future research.
The present study highlighted a strong association between NPO and worry. However, in a recent review of prominent
GAD models (Behar et al., 2009) the IU model (Dugas et al., 1998) was the only one to feature NPO. There are two
possible explanations. The first possibility is that despite being overlooked in many models of GAD, NPO is an
important psychological construct that should be considered in future conceptualisations of worry. The second
possibility is that NPO may not be involved in worry, and current conceptualisations and measures of NPO may in
fact be capturing similar constructs, such as IU. This possibility was examined by the secondary aims of the review
which attempted to first establish a clear definition and understanding of what NPO constitutes and how it has been
measured in the literature, and second to examine the relationship between NPO and related constructs such as IU.
alone construct. Within the worry literature, NPO has been measured in three different ways and papers were divided
into three subgroups accordingly. The first subgroup included measures that defined NPO to relate to problem solving
confidence; the second subgroup, measures that defined NPO as a set of cognitive-emotional variables (The Social
Problem Solving Inventory, SPSI; D’Zurilla & Nezu, 1990; the Social Problem Solving Inventory-Revised, SPSI-R;
Maydeu-Olivares & D’Zurilla, 1996); and the third subgroup, measures that defined NPO as a purely cognitive set of
variables (the Negative Problem Orientation Questionnaire, NPOQ; Gosselin et al., 2005).
The type of NPO measure was a significant moderator of the relationship between NPO and worry. Examination of
the individual subgroups showed that NPO had a significantly stronger correlation with worry, when it was measured
as either a cognitive-emotional, or uniquely cognitive set of items, rather than as purely problem solving confidence.
These findings have important implications for our understanding of NPO.
The first re-definition of NPO in the literature involved NPO being defined as ‘a disruptive cognitive-emotional set of
variables’ (D'Zurilla, Nezu, & Maydeu-Olivares, 1998), rather than simply ‘problem solving confidence’ (Davey,
1994).The review findings indicate that this re-definition led to NPO being more strongly correlated to worry. Thus,
NPO appears to be a broader concept than problem solving confidence alone.
The second redefinition in the literature involved a shift from NPO being defined as a cognitive-emotional set of
variables, to purely a cognitive set of variables. Robichaud and Dugas (2005) argued that the former approach led
to an inability to separate cognitive predispositions towards problem solving scenarios (process), with the emotional
and affective consequences of failure within such scenarios (outcome). They proposed that their Negative Problem
Orientation Questionnaire avoided this limitation by measuring NPO as a purely cognitive construct, and therefore
only captured cognitive predispositions to problem solving scenarios. However, the present study found no significant
difference between measures that conceptualised NPO to be a cognitive-emotional 1 vs. a purely cognitive set of
beliefs. It is therefore questionable whether improving the definition a second time has furthered our understanding
of NPO. On reflection, it is unclear why Robichaud and Dugas (2005) do not consider emotional factors to be involved
in the process of problem solving, or cognitive factors to be involved in the outcome of problem solving attempts.
Thus, it appears that both cognitive and emotional factors may shape individuals predispositions towards problem
solving; further research is now required to clarify their exact contributions to the NPO construct.
Limitations
The review summarized and synthesised the findings of correlational research. Therefore, causality between NPO
and worry cannot be inferred. Further research, including experimental manipulations or longitudinal designs that
1
No differences were identified between studies that used the SPSI (considers NPO to exist on same scale as positive problem
orientation [PPO]) and the SPSI-R (considers NPO to exist on a separate sale to PPO).
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 335
can show temporal precedence, is required to clarify causality between NPO and worry. The present reviewed
included 33 effect sizes. While a large Orwin’s fail safe N indicated that the review’s findings were robust, a larger
number of effect sizes would have allowed potential moderators to be examined in more detail. A consideration for
future research may be to use more relaxed inclusion criteria and include more grey literature. In some of the
moderator analyses the meta-analytic assumption of independence was broken. While this is a weakness of the
current study, it represents a trade-off as this approach increased the power of the study by maximizing the number
of studies included. In addition, effect sizes from the same study were assigned half weights to ensure their
contribution was not overemphasised. Only seven out of the 33 effect sizes used in the study were from clinical
samples. Therefore, the field and the review somewhat neglects clinical populations and instead mainly assessed
the role of NPO in worry within analogue samples. Future research should concentrate on examining NPO within
clinical samples. It is also possible that rather than the review examining differences in the relationship between
somewhat theoretically different subgroups of NPO measures and worry, the review may have simply examined
whether NPO measures have increased in construct validity. Nevertheless, the subsets appeared to have good face
validity and were found to have significantly different relationships to worry. While the present review has begun to
investigate the NPO construct in detail, further research is required to establish the key constructs that underpin
NPO.
Future Directions
The current findings have important implications for future research and clinical practice. On the assumption that
NPO has some causal or maintaining role in worry, the first recommendation is for future worry interventions to
continue to target NPO. Future interventions should 1) continue to target NPO rather than problem solving skill per
se and 2) examine individual’s cognitive and emotional appraisals of problem solving scenarios, rather than simply
their ‘problem solving confidence’. Nevertheless, it remains unclear whether such interventions would be targeting
NPO, or instead a facet of IU. Therefore, an important goal for future research is to disentangle and help clarify the
relationship between these two constructs.
Conclusion
Despite NPO receiving considerable interest in the literature, in a recent review of prominent GAD models, it only
featured in one (Behar et al., 2009). It was therefore timely to review the relationship between NPO and worry. A
random effects model was used to synthesise the findings of 33 effect sizes, and 5,376 participants, to show that
NPO was strongly correlated with worry. Significant heterogeneity was identified between the studies. The strength
of the relationship between NPO and worry was dependent on the type of NPO measure used. The review indicated
that recent NPO measures, which define it to be a belief set that negatively predisposes individuals towards problem
solving situations, were most strongly correlated to worry. However, it remains unclear whether NPO should be
considered as a cognitive, or a cognitive-emotional, construct, and the extent to which NPO and IU are independent
constructs. Clarification of these matters will help improve future theoretical models, which may ultimately lead to
more effective treatment of excessive worry and disorders such as GAD.
References
APA. (1994). Diagnostic and Statistical Manual of Mental Disorders (4th Edition). Washington, DC: American
Psychological Association.
Anderson, R. J. & Goddard, L. (2009). Social Problem-Solving Processes and Mood in College Students: An
Examination of Self-report and Performance-based Approaches. Cognitive Therapy Research, 33, 175-186.
https://doi.org/10.1007/s10608-007-9169-3
Barahmand, U. (2008). Age and gender differences in adolescenborkt worry. Personality and Individual
Differences, 45(8), 778-783. https://doi.org/10.1016/j.paid.2008.08.006
Behar, E., DiMarco, I. D., Hekler, E. B., Mohlman, J., & Staples, A. M. (2009). Current theoretical models of
generalized anxiety disorder (GAD): Conceptual review and treatment implications. Journal of Anxiety
Disorders, 23, 1011-1023. https://doi.org/10.1016/j.janxdis.2009.07.006
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 336
Belzer, K. D., D'Zurilla, T. J., & Maydeu-Olivares, A. (2002). Social problem solving and trait anxiety as predictors
of worry in a college student population. Personality and Individual Differences, 33(4), 573-585.
https://doi.org/10.1016/S0191-8869(01)00173-8
Birrell, J., Meares, K., Wilkinson, A., & Freeston, M. (2011). Toward a definition of intolerance of uncertainty: A
review of factor analytical studies of the intolerance of Uncertainty Scale. Clinical Psychology Review, 31(7),
1198-1208. https://doi.org/10.1016/j.cpr.2011.07.009
Borenstein, M., Hedher L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. West
Sussex, UK: John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470743386
Borkovec, T. D. (1985). Worry: A potentially valuable concept. Behaviour Research and Therapy, 23(4), 481-482.
https://doi.org/10.1016/0005-7967(85)90178-0
Borkovec, T. D., Alcaine, O. M., & Behar, E. (2004). Avoidance theory of worry and generalized anxiety disorder.
In: R. Heimberg, C. Turk, & D. Mennin (Eds.), Generalized anxiety disorder: Advances in Research and
Practice. New York: Guilford Press. https://doi.org/10.1016/0005-7967(83)90121-3
Borkovec, T., Robinson, E., Pruzinsky, T., & Dupree, J. (1983). Preliminary exploration of worry: some
characteristics and processes. Behaviour Research and Therapy, 21, 9-16.
Bottesi, G., Heary, S. Ham, H., Peden, R. & Freeston, M. H. (2012). The intolerance of uncertainty model revisited:
How do the model components relate to each other? Paper presented at the 42nd annual EABCT Congress,
Geneva, Switzerland.
Carleton, N. R., Gosselin, P., & Asmundson, G. J. G. (2010). The Intolerance of Uncertainty Index: Replication and
Extension with an English Sample. Psychological Assessment, 22(2), 396-406.
https://doi.org/10.1037/a0019230
Carleton, N. R. (2012). The intolerance of uncertainty construct in the context of anxiety disorders: theoretical and
practical perspectives. Expert Review of Neurotherapeutics, 12(8), 937-947. https://doi.org/10.1586/ern.12.82
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. New Jersey, NJ: Lawrence Publishers.
Coutu, M. F., Durand, M. J., Marchand. A., Labrecque, M. E., Berbiche, D., & Cadieux, G. (2013). Factors
associated with generalized anxiety in workers undergoing rehabilitation for persistent musculoskeletal pain.
Disability and Rehabilitation, 35(19), 1599-1607. https://doi.org/10.3109/09638288.2012.748833
D’Zurilla, T. J., & Nezu, A. M. (1990). Development and preliminary evaluation of the Problem Solving Inventory.
Psychological Assessment, 2, 156-163. https://doi.org/10.1037/1040-3590.2.2.156
D’Zurilla, T. J., & Sheedy, C. F. (1991). Relationship between social problem-solving ability and subsequent level of
psychological stress in college students. Journal of Personality and Social Psychology, 61(5), 841-846.
https://doi.org/10.1037/0022-3514.61.5.841
D’Zurilla, T. J., Maydeu-Olivares, A., & Kant, G. L. (1998).Age and gender differences in social problem-solving
ability. Personality and Individual Differences, 25(2), 241-252. https://doi.org/10.1016/S0191-8869(98)00029-4
D'Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (1998). Manual for the social problem-solving inventory-revised.
North Yonawanda New York: Multi-health Systems.
Davey, G. C., Hampton, J., Farrell, J., & Davidson, S. (1992). Some characteristics of worrying: Evidence for
worrying and anxiety as separate constructs. Personality and Individual Differences, 13(2), 133-147.
https://doi.org/10.1016/0191-8869(92)90036-O
Davey, G. (1994). Worrying, social problem-solving abilities, and social problem-solving confidence. Behaviour
Research and Therapy, 32(3), 327-330. https://doi.org/10.1016/0005-7967(94)90130-9
Davey, G., Jubb, M., & Cameron, C. (1996). Catastrophic worrying as a function of changes in problem-solving
confidence. Cognitive Therapy and Research, 20(4), 333-344. https://doi.org/10.1007/BF02228037
Davey, G. C., & Levy, S. (1999). Internal statements associated with catastrophic worrying. Personality and
Individual Differences, 26(1), 21-32. https://doi.org/10.1016/S0191-8869(98)00162-7
De Vlieger, P. P., Crombez, G. G., & Eccleston, C. C. (2006). Worrying about chronic pain. An examination of
worry and problem solving in adults who identify as chronic pain sufferers. Pain, 120(1-2), 138-144.
https://doi.org/10.1016/j.pain.2005.10.022
Drews, A., & Hazlett-Stevens, H. (2008). Relationships between irritable bowel syndrome, generalized anxiety
disorder, and worry-related constructs. International Journal of Clinical and Health Psychology, 8(2), 429-436.
Dugas, M., Letarte, H., Rheaume, J., Freeston, M., & Ladouceur, R. (1995). Worry and problem solving: Evidence
of a specific relationship. Cognitive Therapy and Research, 19, 109-120. https://doi.org/10.1007/BF02229679
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 337
Dugas, M. J. (1997). Intolerance of uncertainty and problem orientation in worry. Cognitive Therapy and Research,
21(6), 593-606. https://doi.org/10.1023/A:1021890322153
Dugas, M., Gagnon, F., Ladouceur, R., & Freeston, M. (1998). Generalized anxiety disorder: A preliminary test of a
conceptual model. Behaviour Research and Therapy, 36(2), 215-226. https://doi.org/10.1016/S0005-
7967(97)00070-3
Dugas, M., Marchand, A., & Ladouceur, R. (2005). Further validation of a cognitive-behavioral model of generalized
anxiety disorder: Diagnostic and symptom specificity. Journal of Anxiety Disorders, 19(3), 329-343.
https://doi.org/10.1016/j.janxdis.2004.02.002
Dugas, M., Savard, P., Gaudet, A., Turcotte, J., Laugesen, N., Robichaud, M., & Koerner, N. (2007). Can the
Components of a Cognitive Model Predict the Severity of Generalized Anxiety Disorder? Behavior Therapy,
38(2), 169-178. https://doi.org/10.1016/j.beth.2006.07.002
Dupuy, J. B., & Beaudoin, S. (2001). Worry: daily self-report in clinical and non-clinical populations. Behaviour
Research and Therapy, 39(10), 1249-1255. https://doi.org/10.1016/S0005-7967(01)00011-0
Dupuy, J. B., & Ladouceur, R. (2008). Cognitive processes of generalized anxiety disorder in comorbid generalized
anxiety disorder and major depressive disorder. Journal of Anxiety Disorders, 22, 505-514.
https://doi.org/10.1016/j.janxdis.2007.05.010
Elliott, T., R., Sherwin, E., Harkins, S., W., & Marmarosh, C. (1995).Self-appraised problem solving ability, affective
states, and psychological distress. Journal of Counselling Psychology, 42, 105-115.
https://doi.org/10.1037/0022-0167.42.1.105
Fergus, T., & Wu, K. (2010). Do symptoms of generalized anxiety and obsessive-compulsive disorder share
cognitive processes? Cognitive Therapy and Research, 34(2), 168-176. https://doi.org/10.1007/s10608-009-
9239-9
Fergus, T. A., & Wu, K. D. (2011). Searching for specificity between cognitive vulnerabilities and mood and anxiety
symptoms. Journal of Psychopathology and Behavioral Assessment, 33(4), 446-458.
https://doi.org/10.1007/s10862-011-9245-6
Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry?
Personality and Individual Differences, 17(6), 791-802. https://doi.org/10.1016/0191-8869(94)90048-5
Gentes, E. L., & Ruscio, A. M. (2011). A meta-analysis of the relation of intolerance of uncertainty to symptoms of
generalized anxiety disorder, major depressive disorder, and obsessive-compulsive disorder. Clinical
Psychology Review, 31, 923-933. https://doi.org/10.1016/j.cpr.2011.05.001
Gonzalez Rodriguez, M., Rovella, A., Barbenza, C., & Rausch, L. (2011). Trait worry and generalized anxiety
disorders in adolescents: cognitive processes contributions. Revista Iberoamericana De Diagnostico Y
Evaluacion-E Avaliacao Psicologica, 2(32), 123-142.
Gosselin, P., Dugas, M., & Ladouceur, R. (2002). Worry and problem solving: The role of negative problem
orientation. Journal de Therapie Comportementale et Cognitive, 12(2), 49-58.
Gosselin, P., Ladouceur, R., & Pelletier, O. (2005). Evaluation of an individual's attitude toward daily life problems:
The Negative Problem Orientation Questionnaire. Journal de Therapie Comportementaleet Cognitive, 15(4),
141-153. https://doi.org/10.1016/S1155-1704(05)81235-2
Hedges, L. V., & Olkin, I. (1985).Statistical merthods for meta-analysis. Orlando Florida: Academic Press.
Heppner, P. P., & Petersen, S. (1982). The development and implications of a personal problem-solving inventory.
Journal of Counselling Psychology, 29, 66. https://doi.org/10.1037/0022-0167.29.1.66
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D., G. (2003). Measuring inconsistency in meta-analysis.
British Medical Journal, 327, 557-560. https://doi.org/10.1136/bmj.327.7414.557
Hutcutt, A. I., & Arthur, W. (1995). Development of a new outlier statistic for meta-analytic data. Journal of
Consulting and Clinical Psychology, 74, 401-415.
Kertz, S., & Woodruff-Borden, J. (2013). The role of metacognition, intolerance of uncertainty, and negative
problem orientation in children's worry. Behavioural and Cognitive Psychotherapy, 41(2), 243-248.
https://doi.org/10.1017/S1352465812000641
Khawaja, N. G., & Chapman, D. (2007). Cognitive predictors of worry in a non-clinical population. Clinical
Psychologist, 11(1), 24-32. https://doi.org/10.1080/13284200601151778
Koerner, N., Dugas, M. J., Gosselin, P., & Langlois, F. (2013). A Preliminary Test of the Intolerance of Uncertainty
Model of Generalized Anxiety Disorder. Paper presented at the CACBT 2013 Conference, Montreal, Canada.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 338
Ladouceur, R., Blais, F., Freeston, M. H., & Dugas, M. J. (1998). Problem solving and problem orientation in
generalized anxiety disorder. Journal of Anxiety Disorders, 12(2), 139-152. https://doi.org/10.1016/S0887-
6185(98)00002-4
Ladouceur, R., Dugas, M. J., Freeston, M. H., Rheaume, J., Blais, F., Boisvert, J.-M., & Thibodeau, N. (1999).
Specificity of Generalized Anxiety Disorder Symptoms and Processes. Behavior Therapy, 30(2), 191-207.
https://doi.org/10.1016/S0005-7894(99)80003-3
Ladouceur, R., Gosselin, P., & Dugas, M. J. (2000). Experimental manipulation of intolerance of uncertainty: a
study of theoretical model of worry. Behaviour Research and Therapy, 38, 933-941.
https://doi.org/10.1016/S0005-7967(99)00133-3
Laugesen, N., Dugas, M. J., Bukowski, W. M (2003). Understanding Adolescent Worry: The Application of a
Cognitive Model. Journal of Abnormal Child Psychology, 31(1), 55-64. https://doi.org/10.1023/A:1021721332181
Lewinsohn, P. M., Gotlib, I. H., Lewinsohn, M. A., Seeley, J. R., & Allen, N. B. (1998). Gender differences in anxiety
disorders and anxiety symptoms. Journal of Abnormal Psychology, 107, 109-117. https://doi.org/10.1037/0021-
843X.107.1.109
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications, Inc.
Matthews, A. (1990). Why worry? The cognitive function of anxiety. Behaviour Research and Therapy, 28, 455-
468. https://doi.org/10.1016/0005-7967(90)90132-3
Maydeu-Olivares, A., & D’Zurilla, T. J. (1996). A factor-analytic study of the Social Problem-Solving Inventory: An
integration of theory and data. Cognitive Therapy and Research, 20, 115-133.
https://doi.org/10.1007/BF02228030
McManus, S., Meltzer, H., & Brugha, T. (2009). Adult Psychiatric Morbidity in England, 2007: Results of a
Household Survey. Leeds.
Meeten, F., Dash, S. R., Scarlet, A. L. S., & Davey, G. C. L. (2012). Investigating the effect of intolerance of
uncertainty on catastrophic worrying and mood. Behaviour Research and Therapy, 50, 690-698.
https://doi.org/10.1016/j.brat.2012.08.003
Mennin, D. S., Heimberg, R. G., Turk, C. L., & Fresco, D. M. (2005). Preliminary evidence for an emotion
dysregulation model of generalized anxiety disorder. Behaviour Research and Therapy, 43, 1281-1310.
https://doi.org/10.1016/j.brat.2004.08.008
Nezu, M. (1986). Negative life stress and anxiety – problem-solving as a moderator variable. Psychological
Reports, 58(1), 279-283. https://doi.org/10.2466/pr0.1986.58.1.279
Nezu, M. (2004). Problem Solving and Behavior Therapy Revisited. Behavior Therapy, 35, 1-33.
https://doi.org/10.1016/S0005-7894(04)80002-9
Parkinson, M., & Creswell, C. (2011). Worry and problem-solving skills and beliefs in primary school children.
British Journal of Clinical Psychology, 50(1), 106-112. https://doi.org/10.1348/014466510X523887
Pretorius, T. B. (1992).Problem-solving appraisal in the association of life stress and depression – a South-African
study. Psychological Reports, 71(3), 855-862. https://doi.org/10.2466/pr0.1992.71.3.855
Rausch, L., Rovella, A., de Barbenza, C. M., & Rodriguez, M. G. (2011). Cognitive processes of the generalized
anxiety disorder in adolescents. Psicologia y Salud, 21(2), 215-226.
Robichaud, M., Dugas, M., & Conway, M. (2003). Gender differences in worry and associated cognitive-behavioral
variables. Journal of Anxiety Disorders, 17(5), 501-516. https://doi.org/10.1016/S0887-6185(02)00237-2
Robichaud, M., & Dugas, M. J. (2005). Negative problem orientation (Part 1): psychometric properties of a new
measure. Behaviour Research and Therapy, 43(3), 391-401. https://doi.org/10.1016/j.brat.2004.02.007
Robichaud, M., & Dugas, M. J. (2005). Negative problem orientation (Part II): construct validity and specificity to
worry. Behaviour Research and Therapy, 43, 403-412. https://doi.org/10.1016/j.brat.2004.02.008
Roemer, L., & Orsillo, S. M. (2002). Expanding our conceptualization of and treatment for generalized anxiety
disorder: integrating mindfulness/acceptance-based approaches with existing cognitive behavioural models.
Clinical Psychology: Science and Practice, 9, 54-68. https://doi.org/10.1093/clipsy.9.1.54
Rosental, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
https://doi.org/10.4135/9781412984997
Rovella, A., Gonzalez, M., Penate, W., & Ibanez, I. (2011). Worry trait and generalised anxiety disorder in a sample
of general population: The differential role of intolerance of uncertainty, cognitive avoidance, negative problem
orientation, and meta-beliefs. Revista Argentina De Clinica Psicologica, 20(2), 101-108.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 339
Ruscio, A. M., & Seitchik, A. E. (2007, November). Problem-solving abilities and appraisals associated with severe
worry and GAD. In A. M. Ruscio (Chair), Cognitive vulnerabilities in generalized anxiety disorder: Basic research
findings and implications for practice. Symposium presented at the annual meeting of the Association for
Behavioral and Cognitive Therapies, Philadelphia, PA.
Smith, K. (2012). Negative Problem Orientation, Cognitive Avoidance and Worry among Young Adolescents
(Unpublished Undergraduate Thesis). Newcastle University School of Psychology. Newcastle Upon Tyne.
Spielberger, C. D. (2010). State-Trait Anxiety Inventory. Corsini Encyclopedia of Psychology. South Florida: John
Wiley & Sons.
Sugiura, Y. (1999). Worry: Problem-solving orientation and uncontrollability. Japanese Journal of Educational
Psychology, 47(2), 191-198. https://doi.org/10.5926/jjep1953.47.2_191
Sugiura, Y. (2002). Problem-focused coping strategies and the uncontrollability of thoughts: Mediating role of meta-
cognition. Japanese Journal of Educational Psychology, 50(3), 271-282.
https://doi.org/10.5926/jjep1953.50.3_271
Sugiura, Y. (2004). Detached mindfulness and worry: a meta-cognitive analysis. Personality and Individual
Differences, 37(1), 169-179. https://doi.org/10.1016/j.paid.2003.08.009
Sugiura, Y. (2007). Responsibility to continue thinking and worrying: evidence of incremental validity. Behaviour
Research and Therapy, 45(7), 1619-1628. https://doi.org/10.1016/j.brat.2006.08.001
Szabo, M., & Lovibond, P. F. (2002). The Cognitive Content of Naturally Occurring Worry Episodes. Cognitive
Therapy and Research, 26(2), 167-177. https://doi.org/10.1023/A:1014565602111
Sazbo, M., & Lovibond, P. F. (2004). The Cognitive Content of Thought-Listed Worry Episodes in Clinic-Referred
Anxious and Nonreferred Children. Journal of Clinical Child and Adolescent Psychology, 33(3), 613-622.
https://doi.org/10.1207/s15374424jccp3303_18
Szabo, M., & Lovibond, P. F. (2006). Worry episodes and perceived problem solving: a diary-based approach.
Anxiety, Stress, and Coping, 19(2), 175-187. https://doi.org/10.1080/10615800600643562
Thompson, T., Webber, K., Montgomery, I. (2002). Performance and persistence of worriers and non-worriers
following success and failure feedback. Personality and Individual Differences, 33, 837-848.
https://doi.org/10.1016/S0191-8869(01)00076-9
Wells, A. (1995). Meta-cognition and worry: a cognitive model of generalized anxiety disorder. Behavioural and
Cognitive Psychotherapy, 23, 301-320. https://doi.org/10.1017/S1352465800015897
Wilson, C. C., & Hughes, C. C. (2011). Worry, beliefs about worry and problem solving in young children.
Behavioural and Cognitive Psychotherapy, 39(5), 507-521. https://doi.org/10.1017/S1352465811000269
Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand, 67, 361-
370. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 340
Appendices
Appendix 1: Comparison of NPO-Q and IUS Items
NPO-Q IUS
Similar Items
I see problems as a danger I must get away from all things I am unsure of (I-
IU)
I think that problems will get in the way of things I Feeling unsure stops me from doing most things
need to do
When I have a problem, the first think I do is think When I’m not sure what to do I freeze
that I do not have the ability to solve it
I see problems as a threat to my well-being When I don’t know what will happen, I can’t do
things very well
When I think about all the ways to solve a The smallest worry can stop me from doing things
problem, I still wonder if my answer will work
It often seems like I can’t beat my problems Even if you plan things really well, one little thing
can ruin it
Remaining items
I think my problems as bigger than they really are When things happen suddenly, I get very upset
I often doubt whether I can solve problems It bothers me when there are things I don’t know
Before I try to solve a problem, I think that it is People should always think about what will
hard to solve problems happen next. This will stop bad things from
happening
When I try to solve a problem I often think I can’t I always want to know what will happen to me in
the future
I often feel that my problems cannot be solved I can’t stand it when things happen suddenly
I think my problems will not go away, even if I can I should always be prepared before things happen
think of ways to solve them