Download as pdf or txt
Download as pdf or txt
You are on page 1of 22

Psychopathology Review

PR Volume 4 (2017), Issue 3, 319-340


ISSN 2051-8315 / DOI:10.5127/pr.034313

The Relationship Between Negative Problem Orientation


and Worry: A Meta-Analytic Review

James Bowness Clarkea, b, Michelle Ford b, Sara Hearyb, Jacqui Rodgersa, Mark H.
Freestona, b
a
Newcastle University
b
Northumberland, Tyne and Wear NHS Trust

Abstract
Generalised Anxiety Disorder (GAD) can have a significant detrimental effect on the lives of sufferers. Research into
the psychological process involved in worry has an important role in the prevention and treatment of problems such
as GAD. Negative problem orientation (NPO) has been identified as an important psychological construct involved
in worry (Gosselin, Ladouceur & Pelletier, 2005). The focus of this meta-analytic review was to further our
understanding of the relationship between NPO and worry, and a related construct called intolerance of uncertainty
(IU). The review identified 31 articles reporting correlations between measures of NPO and worry. These articles
detailed a total of 33 independent effect sizes. Using a random-effects model and raw and attenuation-corrected
values, NPO was found to be significantly correlated to worry (r = 0.57 , CI = 0.55-0.60, k = 33, N = 5376). The type
of NPO measure moderated the relationship between NPO and worry. In addition, NPO was found to be significantly
correlated with IU (r= 0.71, CI = 0.69-0.73, k = 14, N = 3072) and more strongly correlated with IU than with worry
(p<.05). The findings have implications for theory, research and practice. Issues in the measurement of NPO are
discussed, and the relationship between NPO and IU is considered.
© Copyright 2017 Textrum Ltd. All rights reserved.
Keywords: Negative Problem Orientation, Worry, Generalized Anxiety Disorder, GAD, Meta-Analysis
Correspondence to: Mark.Freeston@newcastle.ac.uk
Received 11-Oct-2013; received in revised form 05-Jan-2015; accepted 12-Dec-2014
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 320

Table of Contents
Introduction
Generalised Anxiety Disorder
The Intolerance of Uncertainty Model
Negative Problem Orientation
Review objectives
Method
Literature Search
Search terms.
Search strategy.
Inclusion/exclusion criteria.
Selection of studies.
Meta-Analytic Method
Inter-Rater Reliability.
Independent Effect Sizes.
Outliers.
Transformation of r.
Homogeneity.
Publication Bias.
Correction for Attenuation.
Effect Size Coding.
Sources of Heterogeneity.
Moderator Analysis.
Results
Relationship between Negative Problem Orientation and Worry
Consideration of the Exclusion of Outliers.
Moderator Analysis.
Relationship between Negative Problem Orientation and Intolerance of Uncertainty.
Moderator Analysis.
Relationship between Negative Problem Orientation and Intolerance of Uncertainty and Worry
Discussion
Negative Problem Orientation definition and measurement
Relationship between Negative Problem Orientation and Intolerance of Uncertainty
Limitations
Future Directions
Conclusion
References
Appendices

Introduction

Generalised Anxiety Disorder


Generalised Anxiety Disorder (GAD) is characterised by excessive and uncontrollable worry (APA, 1994). Worry is
defined as a chain of thoughts and images, negatively affect-laden and relatively uncontrollable (Borkovec, Robinson,
Pruzinsky, & Dupree, 1983). Individuals who suffer from the condition often experience severe impairment to their
everyday functioning. Epidemiological studies of GAD in the UK estimate its 1-week prevalence rate to be around
4.4% (McManus, Meltzer & Brugha, 2009).To improve treatment of GAD, it is important to understand the
psychological processes associated with the aetiology and maintenance of pathological worry. A review paper
provided a comprehensive overview of five prominent psychological models of GAD (Behar, DiMarco, Hekler,
Mohlman & Staples, 2009). All of the models share the central tenet that GAD is a consequence of the avoidance of
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 321

internal experiences, and propose that individuals with GAD attempt to variously avoid: emotion-laden stimuli, such
as vivid images (The Avoidance Model; Borkovec, Alcaine & Behar, 2004); uncertainty (The IU Model; Dugas,
Gagnon, Ladouceur & Freeston, 1998); emotions (The Emotion Dysregulation Model; Mennin, Heimberg, Turk &
Fresco, 2005); internal experiences (The Acceptance Based Model; Roemer & Orsillo, 2002); or distressing meta-
cognitions about worry (Wells, 1995).
The models have conceptual differences and can be classified into three categories: cognitive models, namely the
Intolerance of Uncertainty (IU) and Meta-Cognitive Model; emotional/behavioural models, namely the Emotion
Dysregulation and Acceptance-Based Model; and integrative models, namely the Avoidance Model. The conceptual
differences have implications for treatments that follow them. For example, treatments that follow the cognitive
models focus on cognitive restructuring through techniques such as thought challenging and behavioural
experiments, and in the case of the IUM model, using simple problem solving with reality-based worries to address
negative problem orientation and intolerance of uncertainty. In contrast, treatments following the emotion and
behavioural models focus on increasing emotional tolerance through techniques such as emotional skills training or
emotional literacy. To further increase treatment efficacy, the key psychological constructs involved in GAD need to
be established.

Figure 1. The Intolerance of Uncertainty Model (Dugas et al., 1998).

The Intolerance of Uncertainty Model


The IU model (Dugas et al., 1998) is a well supported psychological model of GAD. Research has indicated that the
components of the model have been found to distinguish between GAD patients and those diagnosed with other
anxiety disorders (Dugas, Marchand & Ladouceur, 2005; Ladouceur et al., 1999); distinguish between GAD patients
and non-clinical participants (Dugas, et al., 1998; Ladouceur, Blais, Freeston & Dugas, 1998); predict clinical severity
in GAD patients (Dugas et al., 2007); and through experimental manipulation to have a causal impact on levels of
worry (e.g. Ladouceur, Gosselin & Dugas, 2000). The IU model (Dugas et al., 1998) proposes that intolerance of
uncertainty, positive beliefs about worry, cognitive avoidance and negative problem orientation (NPO), are all
involved in worry.
As shown in Figure 1, according to the IU model, individuals with GAD are likely to have a pervasive intolerance of
uncertainty (IU). Although definitions have varied over the years, IU has been defined as essentially a ‘fear of the
unknown’ (Carleton, 2012), and in clinical settings, individuals with GAD are predicted to avoid or react negatively to
uncertain situations or events. Subsequently, they may attempt to reduce uncertainty by engaging in worry, which is
viewed as a process that will help them resolve a current or anticipated future uncertain situation, or reduce their
distress. In addition, the model proposes that individuals with GAD may have a negative approach towards problem
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 322

solving situations, labelled as negative problem orientation (NPO). As a result, they may avoid engaging in problem
solving or engage in a way that does not lead to clear resolution, which subsequently maintains their worry. While
NPO is a key facet of the IU model, it does not feature in the other GAD models reviewed by Behar et al. (2009). It
is therefore timely to review the NPO construct in detail and its relationship with worry.

Negative Problem Orientation


NPO has been defined as ‘a set of beliefs reflecting perceived threat of problems to well-being, doubt concerning
problem-solving ability, and the tendency to be pessimistic about the outcome’ (Robichaud & Dugas, 2005). It has
previously been defined and measured in a number of different ways, and its development as a concept is best
understood with reference to the problem solving and worry literature.
Within the early problem solving literature, research attempted to establish the key factors that shape how individuals
approach and engage in problem scenarios. The Personal Problem Solving Inventory (PSI; Heppner & Petersen,
1982) defined problem-solving confidence (PSC), personal control and approach-avoidance style as the key
processes involved in problem solving. A typical PSC item is ‘When confronted with a problem, I am unsure whether
I can handle the situation’. Most of the early clinical research with the PSI investigated the role of problem solving in
an individual’s responses to negative life events and stress (Nezu, 1986; Pretorius, 1992).
While a number of researchers have attempted to conceptualise the role that problem solving plays in worry (e.g.
Borkovec, 1985; Matthews, 1990), the first significant study in the field was by Davey, Hampton, Farrell and Davidson
(1992), who used the PSI to argue that ‘poor confidence in one’s own problem-solving capabilities could lead to
worry’. The finding that worrying was associated not with poor problem solving skills per se, but rather with one’s
confidence in their problem solving abilities (Davey, 1994), supported this hypothesis. A large number of studies have
subsequently found low problem solving confidence to be positively correlated with worry (e.g. De Vlieger, Crombez
& Eccleston, 2006; Khawaja & Chapman, 2007). Problem solving confidence has been measured using the PSI,
analogue scales (Davey, Jubb & Cameron, 1996), and questionnaires examining meta-cognitions (The Problem-
Solving Related Meta-Cognition Questionnaire; PSRMC; Sugiura, 2002).
Within the problem solving literature, a second group later proposed that problem solving confidence is actually part
of a general response set that predisposes individuals to have either positive or negative responses to problem
situations, known as their ‘problem orientation’ (D’Zurilla & Nezu, 1990). Problem orientation is conceptualised to
exist on a single continuum from positive problem orientation (PPO) to negative problem orientation (NPO), and
includes one’s cognitive, affective and behavioural reactions to problem situations. Problem orientation is measured
by one of the two major scales on ‘The Social Problem Solving Inventory’ (SPSI; D’Zurilla & Nezu, 1990); the scale
is further divided into cognitive, affective and behavioural subscales. Items on the SPSI include: ‘I usually feel
threatened and afraid when I have an important problem to solve’; ‘when my first efforts to solve a problem fail, I get
very angry and frustrated’; and ‘when I have a problem, I usually believe that there is a solution for it’. Problem
orientation has been found to be a better predictor of stress levels in college students than problem solving ability
(e.g. D’Zurilla & Sheedy, 1991), to be correlated with affect (e.g. Elliott, Sherwin, Harkins & Marmarosh, 1995) and
to be associated with worry (e.g. Dugas, Letarte, Rheaume, Freeston & Ladouceur, 1995).
Maydeu-Olivares and D’Zurilla (1996) revisited the SPSI and used factor analysis to propose that rather than existing
on a single continuum, problem orientation was better conceptualised as having both positive (PPO) and negative
(NPO) dimensions, which are distinct, although related processes. The Social Problem Solving Inventory Revised
(SPSI-R; Maydeu-Olivares & D’Zurilla, 1996) divides questions from the SPSI into five scales, two of which are PPO
and NPO. This distinction has led to research into NPO as a stand-alone construct.
Both the SPSI (D’Zurilla & Nezu, 1990) and the SPSI-R (Maydeu-Olivares & D’Zurilla, 1996) conceptualise NPO as
a ‘cognitive-emotional set of variables’. As a result, both questionnaires include items that assess cognitive and
emotional predispositions towards problems. Robichaud and Dugas (2005) argue that this approach is problematic
because an individual’s cognitive appraisals of a problematic situation (process) will be entangled with their emotional
and affective responses to failing to solve a problem (outcome). As a consequence, they propose it is difficult to
capture the psychological factors associated with the process of problem solving, which are independent of factors
associated with the outcome of problem solving attempts. The Negative Problem Orientation Questionnaire (NPO-
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 323

Q; Gosselin, Ladouceur & Pelletier, 2005) tried to rectify this by conceptualising NPO as a purely cognitive
phenomenon. Items on the NPO-Q include: ‘I often have the impression that my problems cannot be solved’, ‘I often
see my problems as bigger than they really are’ and ‘I consider problems to be obstacles that interfere with my
functioning’.
As NPO has moved towards being measured as a purely cognitive psychological process, it raises the question
whether individuals do in fact have a general negative predisposition towards problem solving situations, or whether
they have an aversion to the uncertainty evoked by problem solving scenarios, as captured by the Intolerance of
Uncertainty Scale (IUS, Freeston, Rhéaume, Letarte, Dugas, Ladouceur, 1994).

Review objectives
Within the current literature a number of studies have examined the relationship between NPO and worry.
Nevertheless, given the variability regarding the definition and measurement of NPO the picture remains unclear. It
is therefore timely to review the relationship between NPO and worry, and the impact that the type of NPO measure
has on this relationship.
A secondary aim of the research is to examine the relationship between NPO and IU. Recent developments in the
conceptual understanding of NPO and IU, have led to the NPO-Q (Gosselin et al., 2005) and the IUS (Birrell, Meares,
Wilkinson, & Freeston, 2011) containing similar items (Appendix 1). There is strongest overlap with the factor of the
IUS-12 that has been variously labelled as inhibitory IU (Carleton, 2012) or behavioural paralysis (Birrell et al., 2011).
As a result, it is unclear whether NPO would be best understood as an alternative measure of IU, which makes it
redundant, or an expression of IU, which adds to our understanding of IU and GAD.
Some research has attempted to explore whether NPO is distinct from IU by examining whether NPO still has a
relationship with worry when controlling for IU (Carleton, Gosselin & Asmundson, 2010). Nevertheless there are not
enough studies in this area to draw any firm conclusions. Another way of examining the relationship between NPO
and IU, is to establish the degree of shared variance between the constructs. If the shared variance between IU and
NPO is higher than the relationship between NPO and worry (or IU and worry), this could suggest that NPO and IU
are assessing the same construct, while lower levels may indicate that the constructs have unique features.
While the review aims can be addressed by a number of methodologies, the use of a meta-analysis has a number
of advantages. Firstly, it provides a comprehensive review of the relationship between NPO and worry by including
all available data in the field. Secondly, through the use of correction for attenuation procedures the variance
attributable to measurement error can be factored out of the analysis. And finally, through the partitioning of variance,
the impact of the type of NPO measure as a potential moderator can be examined, helping establish whether NPO
appears to be more accurately conceptualised as a cognitive-emotional, or purely cognitive, variable. While a recent
meta-analysis examined the relationship between IU and worry (Gentes & Ruscio, 2011), this has not been
conducted for NPO and worry, nor NPO and IU.
The review objectives are to:
1. Use all currently available data to establish the mean correlation between NPO and worry.
2. Identify whether the type of NPO measure moderates the relationship between NPO and worry.
3. Use all currently available data to establish the mean correlation between NPO and IU.

Method

Literature Search

Search terms.
Search terms were selected by identifying key words used within the field. Consideration was given to searching for
NPO. However, due to NPO being a relatively new construct, the search strategy was broadened to capture the
whole problem solving literature: the search terms ‘Problem Solving’ and ‘Problem Orientation’ were used. The
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 324

search term ‘Worry’ was used. Due to worry being a central feature of GAD, ‘GAD’ and ‘Generalized Anxiety Disorder’
were also used as search terms. Wild cards (*) were used to ensure that all variations of the search terms were
captured
Negative Problem Orientation: Problem Solv* (solving, solving strategies, solving approaches) OR Problem
Orientation* (Orientation, Orientations)
Worry: Worr* (worry, worrying, worried) OR Generali?ed (Generalised, Generalized) Anxiety Disorder* (Disorder,
Disorders, Disordered) OR GAD

Search strategy.
In addition to database searches, a range of additional strategies were employed. Four prominent authors in the field
(Davey, Dugas, Ladouceur & Ruscio) were contacted via email and asked if they knew of any published or
unpublished data that could contribute to the meta-analysis. Davey was selected due to him being one of the first
authors to look at the relationship between problem-solving, NPO and worry; Dugas and Ladouceur, due to them
being prolific researchers in the field; and Ruscio due to her conducting a meta-analysis in the worry and IU field.
Two additional articles were identified, one of which met inclusion criteria. A number of relevant journals were
searched by hand (Behaviour Research and Therapy, Cognitive Therapy and Research, Journal of Anxiety Disorders
& Behaviour and Cognitive Psychotherapy), no additional literature was identified. The Journal of Anxiety Disorders
was selected due to it being relevant to GAD. The Journal of Behaviour Research and Therapy, Cognitive Therapy
and Research, and Behaviour and Cognitive Psychotherapy were all selected due to them being relevant to cognitive
behaviour therapy (and therefore the IU model of GAD). To increase the chance of capturing grey literature, the
search engines EThOS, WorldCat and ProQuest Dissertation Publishing were searched. No additional literature was
identified. Dugas was identified as a key author in the field. An author search, conducted in Web of Knowledge,
Embase and Psycinfo, produced 55 hits: 17 were identified as duplicates and the remaining 38 articles did not meet
inclusion criteria. The reference list from a review of problem solving was examined (Nezu, 2004). NPO is a relatively
new construct and within the problem solving literature a number of earlier studies have examined similar phenomena
such as problem solving confidence. No additional articles were identified. A citation search of articles citing a key
paper in the area (Dugas et al., 1998) was conducted. The paper had subsequently been cited 172 times. Thirty four
of these articles were duplicates, while the remaining papers (138) did not meet inclusion criteria.

Inclusion/exclusion criteria.
Correlational research that reported a zero order r value between a measure of worry and NPO was included. As
worry is a commonly occurring process present in non-clinical populations (Dupuy & Beaudoin, 2001), research
conducted in both analogue and clinical populations was included. Worry and anxiety have been identified as
separate constructs (Davey et al., 1992). Studies that used anxiety, rather than worry, as a primary outcome measure
were therefore excluded from the review; the State-Trait Anxiety Inventory (Spielberger, 2010) and the Hospital
Anxiety and Depression Scale (Zigmond & Snaith, 1983) were not considered suitable.
Between-groups research that examined NPO in a low versus high worry group was included. Studies were required
to include groups that differed in worry levels, and to report mean NPO scores for each group. GAD is characterised
by chronic and persuasive worry (APA, 1994) and therefore GAD patients were deemed to represent a high worry
group. Between-groups research using other clinical disorders was not included, due to the worry being specific to
one domain (e.g. health anxiety) or acute episodes of worry specific to the main problem (e.g. panic disorder), rather
than persistent general themes.
No papers were initially excluded based on their language of publication. However, papers that could not be
adequately translated despite the help of interpreters (e.g. Japanese) were not included. Both published and
unpublished research, including a conference presentation and a doctorate thesis, was included in the review.
Theoretical and conceptual reviews were excluded. No studies were excluded based on their date of publication, the
earliest publication dates varied according to the database used. No studies were excluded on the basis of age of
participants. The inclusion criteria aimed to increase the coverage of the review.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 325

Figure 2. Flow diagram for inclusion and exclusion in meta-analysis.

Selection of studies.
A total of 1552 papers were identified through the literature search. Six hundred and fifty seven were identified as
duplicates and removed, leaving a total of 895. Identified papers were reviewed at the level of title and abstract
against the inclusion/exclusion criteria; 852 papers did not meet inclusion criteria and were removed, leaving a total
of 43 papers. Two additional papers in the process of being published were identified through the research team
(Smith, 2012; Bottesi, Heary, Ham, Peden & Freeston, 2012). Two papers were identified through author contact
(Meeten, Dash, Scarlet & Davey, 2012; Ruscio & Sietchik, 2007). A full text review of the 47 studies was conducted
to determine whether the studies reported the necessary statistics, used an appropriate measure of NPO and worry,
and could be adequately translated. This text review revealed that eight papers did not report the appropriate statistics
(Carleton et al., 2010; Drews & Hazlett-Stevens, 2007; Dugas et al., 1998; Dugas et al., 1995; Dupuy & Ladouceur,
2008; Kertz & Woodruff-Borden, 2013; Laugesen, Dugas & Bukowski, 2003; Meeten et al., 2012), four papers did
not include an adequate measure of NPO (Szabo & Lovibond, 2002, 2004 & 2006; Thompson, Webber &
Montgomery, 2002), two did not use an adequate worry measure (Anderson & Goddard, 2009; Robichaud & Dugas,
2005) and two could not be adequately translated (Sugiura, 1999, 2002). This left a total of 31 papers. One paper
used a within-group design to examine the relationship between NPO and worry in a sample of chronic pain patients
(De Vlieger et al., 2006). The worry and NPO measures used were not pain specific. While the relationship between
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 326

NPO and worry may have been different within certain clinical populations, including this study increased the
coverage of the study. No other papers conducted in specific clinical populations met inclusion criteria and were
therefore not included. The paper selection process is outlined below in Figure 2.

Meta-Analytic Method
The overall meta-analytic method was modelled on Gentes and Ruscio (2011) which uses a standard and thorough
approach (based on Borenstein, Hedher, Higgins & Rothstein, 2009) and addressed a similar question to the present
review (the relationship between a psychological construct, intolerance of uncertainty, and worry or GAD). All primary
sources for formulae were obtained and are referenced at the appropriate point in the following paragraphs. Formulae
were input into spreadsheets, and data from each of the primary sources articles were tested to ensure accuracy.
The formulae will not be repeated here as they are reported in detail by Gentes and Ruscio (2011). Excel was used
for data extraction and synthesis using spreadsheets developed by the last author (MHF).

Inter-Rater Reliability.
To check for inter-rater reliability, relevant data were independently coded into separate spreadsheets by the first
(JBC) and third (SH) authors of the study. There was a 94% agreement rate; discrepancies were primarily identified
in papers that were not written in English. All differences were reviewed by the author and second researcher with
reference to original papers. This procedure led to a 100% agreement on all effect sizes and study characteristics
detailed in the study.

Independent Effect Sizes.


Meta-analysis is based on the assumption of independence of each observation (Rosental, 1991). Some studies
reported multiple independent samples; each sample was coded and included as an individual observation. To avoid
overlapping samples or double counting the data, the sample characteristics of each study were checked to ensure
that data from the same source had not been inadvertently included twice in studies by the same group of authors.
One study (Sugiura, 2007) reported two problem orientation measures that had been conducted in partially
overlapping samples. In this case, the sample with the less established problem solving measure was dropped
(Problem Solving Related Meta-Cognitions).
A number of studies reported multiple well-established worry measures. In these cases, average correlations were
calculated for each study and reported as a single statistic. One study (Belzer, D'Zurilla, & Maydeu-Olivares, 2002)
reported both the PSWQ and a less established measure, the catastrophic worry questionnaire (CWQ). In this
instance, the CWQ was dropped and only the PSWQ was reported. Likewise, a number of studies reported two
measures of problem orientation which were correlated with a single measure of worry; the coefficients were
averaged.
Although the assumptions of independence were met for the main analyses, the independence assumption was
relaxed for examining moderators. There is a trade-off between power to test for moderators (which result in
comparisons of subgroups that are by definition smaller), and respecting the assumption of independence for studies
that reported effect sizes for more than one measure of a construct. In these cases, if there were two effect sizes
from the same study, they were assigned half weights to ensure that their contribution was not overemphasized. This
approach aimed to increase the power of the study by maximizing k (the number of effect sizes) for moderator
comparisons, while ensuring that the number of participants (N) was not doubled. Effect sizes were interpreted with
reference to Cohen (1988) (small = 0.10, medium= 0.30, large =0.50).

Outliers.
The sample-adjusted meta-analytic deviance (SAMD) statistic (Hutcutt & Arthur, 1995) was calculated to test for the
presence of statistical outliers. The distribution of SAMD statistics approximates a normal t distribution (Hutcutt &
Arthur, 1995). It was important to note that elimination of outliers may lead to the underestimation of true variability,
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 327

while failure to eliminate outliers may lead to overestimation of variability (Huffcutt & Arthur, 1995). Examination of
the distribution of the coefficients highlighted candidates for Winsorization.

Transformation of r.
The weighted mean effect size of each study was calculated. Due to correlation coefficients having a skewed
standard error formulation (Rosental, 1991), Fisher’s Zr transform was used to transform effect sizes. Effect sizes
were then weighted by the inverse of the squared standard error of the Z-transformed correlation.

Homogeneity.
The homogeneity of the data was analysed using the Q statistic. The Q statistic is distributed as a Chi square with k
– 1 degrees of freedom (k is the number of effect sizes; Hedges & Olkin, 1985). The heterogeneity of the data was
examined using the I2 statistic (Higgins, Thompson, Deeks & Altman, 2003). Benchmarks for low medium and high
heterogeneity, of 25%, 50% and 75% respectively are outlined by Higgins et al. (2003).

Publication Bias.
Referring to the phenomena of papers with significant findings having a higher chance of publication, publication bias
can lead to the over reporting of effect sizes in the literature. Orwin’s fail safe N (FSN), indicates the minimum number
of unpublished findings with an effect size of no interest (r = .10) that would need publication to reduce the mean
observed effect size to zero. Funnel plots were generated but due to the small number of studies they were not very
informative.

Correction for Attenuation.


Corrected mean effect sizes were estimated. Reliability coefficients (alpha) for NPO and worry measures were
extracted in order to correct for measurement unreliability. When alpha was not reported, if possible it was estimated
using the mean reliability for the measure, as reported by other studies in the meta-analysis. If other studies in the
meta-analysis had not used the measure, the estimate was made using the alphas reported in the scale’s
development. Where studies had used two worry measures an average alpha coefficient was calculated. For between
group studies, group difference, rather than a worry measure, was used as the independent variable. In these
instances, the average alpha coefficient for all measures used in the meta-analysis was used. This correction
increases sampling error, and therefore inverse variance weights were also corrected for attenuation. The same
method was applied to the IU measures.

Effect Size Coding.


Thirty one studies were included in the meta-analysis. Two of the studies reported two independent effect sizes,
giving a total of 33 independent effect sizes. Study characteristics for each effect size were coded using the following
information: language of paper, publication date, nationality of sample, sample size, sample type, mean age (and
SD), percentage of female participants, zero-order correlation between NPO measure and worry and IU measure,
and reliability coefficients (alpha) for all measures.
Within the literature there is no consensus on a gold standard measure of NPO. The study aimed to examine NPO
and its evolution in more detail and therefore all measures were treated equally. When studies reported separate
correlations for male and female participants, mean correlations weighted by the proportion of each gender were
calculated. Five studies were included that used a between-group design. Effect sizes were calculated by using the
standardized mean difference (Cohen’s d) and converted to a point biserial correlation weighted by the proportion of
participants in each group.

Sources of Heterogeneity.
Two statistical models are used for meta-analysis, the fixed and random-effects models. The fixed-effects model
assumes that there is one true effect size which underlies all studies in the analysis and any observed differences
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 328

are attributable to sampling error; findings are typically generalizable to studies included in the review. The random-
effects model posits that true effect size may vary from study to study; findings are generalizable beyond studies
included in the review (Borenstein et al., 2009). Due to the review attempting to synthesize research that measured
and conceptualised NPO in different ways, a random effects model was chosen.

Moderator Analysis.
To examine the type of NPO measure as a possible moderator, papers were divided into subsets that had used
theoretically similar NPO measures. The three subsets were initially identified by the first author of the paper and
then reviewed by the last author. It was agreed that these groups represented theoretical shifts in how NPO has been
conceptualised within the literature. The first subgroup, included studies that measured NPO using a measure of
problem solving confidence (e.g. the PSI; Heppner & Petersen, 1982); the second, studies that captured NPO using
a version of the SPSI (D’Zurilla & Nezu, 1990) or the SPSI-R (Maydeu-Olivares & D’Zurilla, 1996), which both
conceptualise NPO as a cognitive, emotional and behavioural construct; and the third, studies that conceptualised
NPO as a cognitive construct (The NPO-Q; Gosselin et al., 2005). Both the first author and a second researcher
independently assigned the papers to the same groups, indicating that the process was reliable.
The review considered the impact of additional moderators, which were participant age and gender.
Age was considered as a possible moderator. While the majority of NPO research has been conducted in adult
samples, more recent research has begun to examine the role of NPO in worry, within child and adolescent samples
(e.g. Parkinson & Creswell, 2011; Wilson & Hughes, 2011). These studies have reported high correlations between
NPO and worry.
Gender was also considered as a moderator. Women have been found to be more likely to report higher levels of
worry (Lewinsohn, Gotlib, Lewinsohn, Seeley & Allen, 1998) and NPO (D’Zurilla, Maydeu-Olivares & Kant, 1998)
compared to men. Thus, while men may under-report their levels of NPO and worry, women are perhaps more likely
to accurately report if they are experiencing high levels of NPO and worry. Differential reporting by men and women
and any associated restricted range effects could influence the strength of correlations, so the percentage of women
in the sample was considered as a moderator.

Results

Relationship between Negative Problem Orientation and Worry


To examine the relationship between NPO and worry, effect sizes were extracted from all identified papers. A
summary of selected studies and the extracted effect sizes are detailed in Table 1and Table 2 respectively.

Table 1: Summary of Studies Included in the Meta-Analysis


Author Sample Type N Mean Age r SE NPO Measure Worry Measure
Davey (1992) Study 1 Analogue 105 22.00 0.30 0.10 PSI SWS
Davey (1992) Study 3 Analogue 94 37.00 0.26 0.10 PSI SWS
Davey (1994) Analogue 82 40.10 0.53 0.11 PSI PSWQ & WDQ
Davey (1996) Analogue 40 21.60 0.39 0.16 Analogue scale Interview
Davey (1999) Analogue 100 29.75 0.62 0.10 CSQ PSWQ
De Vlieger (2006) Chronic Pain Patients 185 54.12 0.42 0.07 PSI WDQ
Khawaja (2007) Analogue 96 25.59 0.56 0.10 PSI PSWQ
Ladouceur (1998) Analogue vs GAD 21.5* 30.35 0.64 0.222 PSI PSWQ
Ladouceur (1999) Analogue vs GAD 34* 37.09 0.53 0.17 PSI PSWQ
Parkinson (2011) Analogue 60 9.67 0.67 0.13 PSI-Child PSWQ-C
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 329

Sugiura (2004) Analogue 127 19.30 0.46 0.09 PSRM PSWQ


Sugiura (2007) Analogue 122 19.50 0.54 0.09 PSRM PSWQ
Sugiura (2007) Study 2 Analogue 77 19.58 0.33 0.11 PSI PSWQ
Wilson (2011) Analogue 57 7.55 0.32 0.13 PSI-Child PSWQ-C
Belzer (2002) Analogue 355 18.85 0.60 0.05 SPS-R-Short PSWQ
Dugas (1997) Analogue 285 23.50 0.61 0.06 SPSI-abridged PSWQ
Dugas (2005) GAD vs PD 45 38.62 0.49 0.15 SPSI-abridged PSWQ
Dugas (2007) ab GAD 84 37.76 0.29 0.11 SPS-R-Short PSWQ
Gosselin (2002) Analogue 346 22.60 0.63 0.05 SPSI-R PSWQ
Gosselin (2005) b Analogue 172* 21.72 0.79 0.08 SPSI-R PSWQ
Ladouceur (1998) Analogue vs GAD 21.5* 30.35 0.50 0.22 SPSI PSWQ
Ladouceur (1999) Analogue vs GAD 34* 37.09 0.63 0.17 SPSI PSWQ
Robichaud (2003) Analogue 217 22.35 0.66 0.07 SPS-R-Short PSWQ & WDQ
Rovella (2011) Analogue 281 30.70 0.61 0.06 SPSI-R PSWQ
Barahmand (2008) b Analogue 197 17.49 0.38 0.07 NPOQ WAQ
Bottesi (2013) Analogue 101 19.39 0.65 0.10 NPOQ PSWQ
Coutu (2013) Musculoskeletal pain 36 - 0.42 0.17 NPOQ WAQ
Fergus (2010) Analogue 414 19.00 0.59 0.05 NPOQ PSWQ
Fergus (2011) Study 1 Analogue 362 19.50 0.56 0.05 NPOQ PSWQ
Fergus (2011) Study 2 Analogue 363 19.50 0.57 0.05 NPOQ PSWQ
Gonzalez (2011) Analogue 102 16.50 0.62 0.10 NPOQ PSWQ-C
Gosselin (2005) Analogue 172* 21.72 0.65 0.08 NPOQ PSWQ
Rausch (2011) Analogue 102 16.00 0.62 0.10 NPOQ PSWQ-C
Robichaud (2005) Analogue 148 23.60 0.55 0.08 NPOQ PSWQ
Ruscio (2007) Analogue 126 - 0.64 0.09 NPOQ PSWQ
Smith (2012) b Analogue 212 12.85 0.81 0.07 NPOQ-C PSWQ
a= Removed as an outlier in first stage; b = Removed as a outlier in second stage; * = Assigned half weights; Analogue = healthy

control participants; GAD = Generalized Anxiety Disorder; PD = Panic Disorder; PSI = Personal Solving Inventory; PSRM-C =
Problem Solving Related Meta-Cognitive Questionnaire; SPSI = Social Problem Solving Inventory; SPSI-R = Social Problem
Solving Inventory Revised; NPOQ = Negative Problem Orientation Questionnaire; PSWQ = Penn State Worry Questionnaire;
PSWQ-C = Penn State Worry Questionnaire-Child Version; WAQ = Worry and anxiety questionnaire; WDQ = Worry Domains
Questionnaire; SWS = Student Worry Scale.

Table 2: Random Weighted Effect Sizes between NPO and Worry


Random effects model

n k Mean r 95% CI Q I2% FSN Mean rho 95% CI

All studies 5376 33 0.57 0.55-0.60 94.07** 65.98 157 0.65 0.62-0.67

PSC 1200.5 14 0.47 0.41-0.52 22.19* 41.42 51 0.55 0.49-0.60

SPSI 1485.5 9 0.61 0.57-0.65 16.77** 52.31 46 0.71 0.66-0.75

NPOQ 2335 12 0.59 0.56-0.61 23.21** 52.62 58 0.64 0.61-0.67

*= Significant at the level of 0.05; ** = Significant at the level of 0.01; N = Number of participants; k = Number of papers; CI =
Confidence Intervals; FSN =
Orwin’s fail safe N. Note included studies assigned half weights resulting in some N values including decimal points.

The overall mean r between NPO and worry was 0.57 (95% IC = .55-.60) (Table 2). Orwin’s fail safe N indicated that
an additional 157 studies with an effect size of no interest would need to be published in order to reduce the mean
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 330

correlation to zero. As shown in Table 2, Q values revealed that the effect size distributions were significantly
heterogeneous (p<0.01). Consideration was given to removing outliers in the dataset.

Consideration of the Exclusion of Outliers.


Initial examination of the SAMD statistic indicated a large number of potential outliers. Examination of the distribution
of the coefficients suggested that three values (two high and one low) were detached from the distribution and were
candidates for Winsorization. Winsorizing the outliers to 2SDs of the mean would have resulted in them becoming
more extreme and, therefore, a simple “end of the distribution +/- .01” rule was adopted. Hence, values of .81 (Smith,
2012) and .72 (Gosselin et al., 2005) became .69 and .68 respectively, and .26 (Davey et al., 1992) became .28. The
SAMD statistics were once again examined and there were still significant outliers, mostly at the lower end of the
distribution. On inspection it was clear that these were due to negative skew, rather than true outliers. Winsorizing
the three values led to a reduction in heterogeneity from Q = 94.07 to 79.03, and I2 from 65.98 to 59.51. Sensitivity
analysis was conducted by trial sequential removal of the most extreme outliers but with up to four outliers removed,
heterogeneity remained significant and I2 dropped to 44.71. We concluded that heterogeneity was a feature of the
data set, and the negatively skewed distribution contributed to the detection of apparent outliers using SAMD. In sum
all effect sizes were retained. The same procedure was conducted for the moderator analysis; 0.81 (Smith, 2012),
0.79 (Gosselin et al., 2005, NPO measure) and 0.26 (Davey et al., 1992) became 0.69, 0.68 and 0.28 respectively.
For the IU and NPO analysis 0.40 (Dugas et al., 2005) became 0.60.

Moderator Analysis.
Moderators within the data were examined by partitioning the heterogeneity of the effect sizes. Studies were
separated into the subgroups previously described. This allowed the within and between group heterogeneity of each
group of effect sizes to be examined (Lipsey & Wilson, 2001). Heterogeneity was then partitioned by separating the
total homogeneity statistic (Qtotal), into the proportion explained by the categorical between groups variable (Qb), and
the residual pooled within groups portion (Qw) (computed by calculating and then summing Q for each group of effect
sizes). The following formula was used:
Qb = Qtotal - Qw
The Q is distributed as chi square. The degrees of freedom for Qb were calculated by the number of groups (j) – 1.
The degrees of freedom for Qw were calculated by k-j, were k equals the number of effect sizes. The portioned
variance is outlined in Table 3 below.

Table 3: Moderation of the Relationship between NPO and Worry, by NPO Measure
Q QPSC QPO QNPOQ Qw df (Qw) p (Qw) I2 (Qw) Qb df (Qb) p (Qb) I2% (Qb)

Random 94.07 22.19 16.77 23.21 62.17 11 0.00 82.31 31.90 2 0.00 93.73
Effects
Model
PSC = Problem solving confidence measures; PO = Problem orientation measures; NPOQ = Negative Problem Orientation
Questionnaire; df = Degrees of freedom; Qw = Q statistic within groups; Qb = Q statistic between groups

As shown in Table 3, a large Q value indicated that there was variability between studies above that attributable to
measurement error. Measure was a significant moderator (Qb). Within group heterogeneity (Qw) was identified,
indicating that even when partitioned into measure specific subgroups, the studies still demonstrated significant within
subgroup variability.
The mean correlation between NPO and worry was established for each subgroup of studies. A mean correlation of
r = 0.47 (95% CI = .41-.52) was identified for studies that used a measure of problem solving confidence; r = 0.61
(95% CI = 0.57-0.65) for studies that used a measure of problem solving orientation; and r = 0.59 (0.56-0.61) for
studies that used the Negative Problem Orientation Questionnaire (NPOQ; Gosselin et al., 2005).
The confidence intervals between the subgroups of studies using a measure of problem orientation and the NPOQ
overlapped. However, there was no overlap between studies using a measure of problem solving confidence and
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 331

other measures. It could therefore be asserted, with a 95% degree of confidence, that the correlations between these
groups were different. In summary, studies using a measure of problem solving confidence had a weaker correlation
with worry compared to studies using a measure of problem orientation or the NPOQ.
The influence of additional moderators in the data was considered. Average participant age and the percentage of
female participants, were correlated with the study NPO-worry correlation. For the mean age of participants, r = -0.33
(-0.62 - -0.01) indicating that as participant age increased, the strength of the relationship between NPO and worry
decreased. For percentage of female participants r = 0.16 (-0.18 – 0.48), but the CIs includes 0, indicating that the
relationship was not significant.

Relationship between Negative Problem Orientation and Intolerance of


Uncertainty.
Fourteen studies examined the relationship between NPO and IU as detailed below in Table 4.
The effect sizes of these studies were pooled together and found to be significantly heterogeneous, as indicated by
a large Q value. Studies were separated into subgroups dependent on the measure of NPO they implemented: the
first included studies that used a measure of problem orientation, and the second, studies that used a measure of
NPOQ. Study characteristics are detailed below in Table 5.

Table 4: Summary of Studies Included in the Meta-Analysis Examining the Relationship between NPO and IU
Sample Mean SE NPO IU
Author Type N Age r Measure Measure
SPSI-
Dugas (1997) Analogue 285 23.50 0.65 0.06 abridged IUS
SPSI-
Dugas (2005) GAD vs PD 45 38.62 0.40 0.15 abridged IUS
Dugas (2007) ab GAD 84 37.76 0.61 0.11 SPS-R-Short IUS
Gosselin (2002) Analogue 346 22.60 0.69 0.05 SPSI-R IUS
Gosselin (2005) b Analogue 172* 21.72 0.67 0.08 SPSI-R IUS
Robichaud (2003) Analogue 217 22.35 0.65 0.07 SPS-R-Short IUS
Barahmand (2008) b Analogue 197 17.49 0.66 0.07 NPOQ IUS
Bottesi, Heary, Ham, Peden & Freeston,
2012) Analogue 101 19.39 0.64 0.10 NPOQ IUS
Fergus (2010) Analogue 414 19.00 0.82 0.05 NPOQ IUS
Fergus (2011) Study 1 Analogue 362 19.50 0.70 0.05 NPOQ IUS
Fergus (2011) Study 2 Analogue 363 19.50 0.71 0.05 NPOQ IUS
Gosselin (2005) Analogue 172* 21.72 0.73 0.08 NPOQ IU
Rausch (2011) Analogue 102 16.00 0.74 0.10 NPOQ IUS
Smith (2012) b Analogue 212 12.85 0.79 0.07 NPOQ-C IUSC
a= b
Removed as an outlier in first stage; = Removed as a outlier in second stage; * = Assigned half weights; Analogue = healthy
control participants; GAD = Generalized Anxiety Disorder; PD = Panic Disorder; PSI = Personal Solving Inventory; PSRM-C =
Problem Solving Related Meta-Cognitive Questionnaire; SPSI = Social Problem Solving Inventory; SPSI-R = Social Problem
Solving Inventory Revised; NPOQ = Negative Problem Orientation Questionnaire; IUS = Intolerance of Uncertainty Scale; IU=
Intolerance of Uncertainty questionnaire; IUSC = Intolerance of Uncertainty Scale for Children.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 332

Table 5: Random Weighted Effect Sizes between NPO and IU


Random effects model

n k Mean r 95% CI Q I2% FSN Mean rho 95% CI

All studies 3072 14 0.71 0.69-0.73 35.44* 63.32 86 0.78 0.76-0.80

PO 1149 6 0.66 0.63-0.69 2.23 0.00 34 0.74 0.71-0.77

NPO 1923 8 0.74 0.71-0.76 21.59 67.57 51 0.80 0.77-0.82

N = Number of participants; k = Number of papers; CI = Confidence Intervals; FSN = Orwin’s fail safe N.

As shown by Table 5, both subgroups were found to be homogeneous. While the Q value for all studies substantially
decreased, it was still significant; the influence of the type of NPO measure as a moderator was examined.

Moderator Analysis.
As highlighted by Table 6, a large Q value indicated that there was variability between studies above that attributable
to measurement error
The moderator effect (QB) was significant, indicating that measure was a significant moderator of the relationship
between NPO and IU. Within group heterogeneity (Qw) was not significant, indicating that when controlling for the
type of NPO measure as a moderator the studies were homogeneous.
Reported correlation between negative problem orientation and intolerance of uncertainty. The overall mean r
between NPO and IU was reported to be 0.71 (0.69-0.73) (Table 4). For studies that used a measure of problem
orientation the average correlation with IU was 0.66 (0.63-0.69), and for studies using the NPOQ, r = 0.74 (0.71-
0.76). There was no overlap between the 95% confidence intervals for studies using a measure of problem orientation
and studies using the NPOQ. Thus, the correlations between NPO and IU for these subgroups were different.
Nevertheless, it is worth noting that the power of this finding is relatively low with a relatively small k and relatively
wide confidence intervals.

Table 6: Moderation of the Relationship between NPO and IU, by NPO Measure
Q QPO QNPOQ Qw df (Qw) p (Qw) I2 (Qw) Qb df (Qb) p (Qb) I2% (Qb)
Random 35.44 2.23 21.59 23.82 12 0.02 49.62 11.62 2 0.00 82.79
Effects
Model
PO = Problem orientation measures; NPOQ = Negative Problem Orientation Questionnaire; df = Degrees of freedom; Qw = Q
statistic within groups; Qb = Q statistic between groups

Relationship between Negative Problem Orientation and Intolerance of


Uncertainty and Worry
Examination of the confidence intervals highlight that there was no overlap between the correlation between NPO
and worry (r = 0.57, 0.55-0.60) and NPO and IU (r = 0.71, 0.69-0.73) and the correlation with IU is thus significantly
greater than with worry. Nevertheless, this finding should be interpreted with caution as the relationship between
NPO and IU is based on a relatively small k.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 333

Discussion
The primary aim of the review was to use all currently available data to examine the role of negative problem
orientation (NPO) in worry. The review synthesised research that had conceptualised NPO from somewhat different
perspectives. The identified literature was found to be significantly heterogeneous and a random-effects model was
deemed the most appropriate method to analyse the data. An overall mean correlation of 0.57 (0.55-0.60) between
NPO and worry was identified, when corrected for attenuation, rho = 0.65 (0.62-0.67). In summary, the review found
that NPO is highly correlated with worry.
A second aim of the research was to examine factors that may moderate the relationship between NPO and worry.
Through partitioning variance, the type of NPO measure was identified as a significant moderator. Earlier research
that attempted to assess NPO by measuring problem solving confidence was moderately correlated to worry (r =
0.47, 0.41-0.52), while later research that conceptualised NPO as a stand-alone construct was strongly correlated to
worry (r = 0.59, 0.56 - 0.61). Within the recent literature, no difference was identified between measures that
conceptualised NPO as a cognitive-emotional (r = 0.61, 0.57 - 0.65), compared to a purely cognitive (r = 0.59, 0.56-
0.61) construct. A regression analysis indicated that participant age and gender should be considered as potential
moderators in future research, despite neither of them accounting for significant unique variance in the present
review.
A third aim of the review was to examine the relationship between NPO and intolerance of uncertainty (IU), defined
as ‘a fear of the unknown’ (Carleton, 2012). It is possible that rather than measuring negative predispositions to
problem solving scenarios per se, NPO measures may instead measure aversion to the uncertainty created by
problem solving scenarios (IU). The review found that these NPO and IU were highly correlated (r = 0.71, 0.69-0.73),
when corrections were made for attenuation, rho = 0.78 (0.76-0.80). NPO had a stronger correlation with IU than with
worry. Measure did not moderate these relationships.
Relationship between Negative Problem Orientation and Worry
The present review found that NPO is strongly correlated to worry. This is consistent with the IU model (Dugas et al.,
1998), which proposes that NPO plays an important role in the maintenance of worry. In order to understand the role
that NPO plays in worry it is worthwhile examining the IU model in more detail. The first conceptualisation of the IU
model (Dugas et al., 1998) proposes that one of the outcomes of worry was NPO, which in turn maintains worry. This
indicates that NPO is activated by worry and subsequently plays a key role in its maintenance. Recent revisions of
the IU model propose that the relationship between IU and worry is mediated by NPO, positive beliefs about worry
(PBW), cognitive avoidance (CA; Bottesi, Heary, Ham, Peden & Freeston, 2012) and a negative interpretation bias
(NIB; Koerner, Dugas, Gosselin & Langlois, 2013). In the latter revision of the IU model, these mediators are proposed
to have a sequential relationship, with IU initially activating NPO and PBW, which in turn prompts CA and NIB
(Koerner et al., 2013). This development proposes a subtly different role for NPO by proposing that it is initially
activated by IU, and then in turn activates other mediators involved in the worry process. While it was beyond the
scope of the present review, and the currently available published data, to clarify the role that NPO plays in worry,
these developments highlight an area of interest for future research.
The present study highlighted a strong association between NPO and worry. However, in a recent review of prominent
GAD models (Behar et al., 2009) the IU model (Dugas et al., 1998) was the only one to feature NPO. There are two
possible explanations. The first possibility is that despite being overlooked in many models of GAD, NPO is an
important psychological construct that should be considered in future conceptualisations of worry. The second
possibility is that NPO may not be involved in worry, and current conceptualisations and measures of NPO may in
fact be capturing similar constructs, such as IU. This possibility was examined by the secondary aims of the review
which attempted to first establish a clear definition and understanding of what NPO constitutes and how it has been
measured in the literature, and second to examine the relationship between NPO and related constructs such as IU.

Negative Problem Orientation definition and measurement


The review identified that within the literature NPO has been measured in somewhat different ways. This is
unsurprising given that NPO evolved from the problem solving literature before it was conceptualised as a stand-
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 334

alone construct. Within the worry literature, NPO has been measured in three different ways and papers were divided
into three subgroups accordingly. The first subgroup included measures that defined NPO to relate to problem solving
confidence; the second subgroup, measures that defined NPO as a set of cognitive-emotional variables (The Social
Problem Solving Inventory, SPSI; D’Zurilla & Nezu, 1990; the Social Problem Solving Inventory-Revised, SPSI-R;
Maydeu-Olivares & D’Zurilla, 1996); and the third subgroup, measures that defined NPO as a purely cognitive set of
variables (the Negative Problem Orientation Questionnaire, NPOQ; Gosselin et al., 2005).
The type of NPO measure was a significant moderator of the relationship between NPO and worry. Examination of
the individual subgroups showed that NPO had a significantly stronger correlation with worry, when it was measured
as either a cognitive-emotional, or uniquely cognitive set of items, rather than as purely problem solving confidence.
These findings have important implications for our understanding of NPO.
The first re-definition of NPO in the literature involved NPO being defined as ‘a disruptive cognitive-emotional set of
variables’ (D'Zurilla, Nezu, & Maydeu-Olivares, 1998), rather than simply ‘problem solving confidence’ (Davey,
1994).The review findings indicate that this re-definition led to NPO being more strongly correlated to worry. Thus,
NPO appears to be a broader concept than problem solving confidence alone.
The second redefinition in the literature involved a shift from NPO being defined as a cognitive-emotional set of
variables, to purely a cognitive set of variables. Robichaud and Dugas (2005) argued that the former approach led
to an inability to separate cognitive predispositions towards problem solving scenarios (process), with the emotional
and affective consequences of failure within such scenarios (outcome). They proposed that their Negative Problem
Orientation Questionnaire avoided this limitation by measuring NPO as a purely cognitive construct, and therefore
only captured cognitive predispositions to problem solving scenarios. However, the present study found no significant
difference between measures that conceptualised NPO to be a cognitive-emotional 1 vs. a purely cognitive set of
beliefs. It is therefore questionable whether improving the definition a second time has furthered our understanding
of NPO. On reflection, it is unclear why Robichaud and Dugas (2005) do not consider emotional factors to be involved
in the process of problem solving, or cognitive factors to be involved in the outcome of problem solving attempts.
Thus, it appears that both cognitive and emotional factors may shape individuals predispositions towards problem
solving; further research is now required to clarify their exact contributions to the NPO construct.

Relationship between Negative Problem Orientation and Intolerance of


Uncertainty
An important question for the review was whether NPO is independent of IU, or whether they in fact assess similar
constructs. The review identified a strong relationship between NPO and IU, stronger even than the relationship
between NPO and worry. The type of measure did not moderate the strength of the relationship. However, it is worth
noting that these findings may be underpowered due to a small k (number of studies) and relatively large confidence
intervals.
These findings raise important questions about NPO as a construct. One possibility is NPO may be capturing aversion
to uncertainty, rather than problem solving per se. Thus, NPO may better be understood as a facet of IU, rather than
a stand-alone concept. A second possibility is that despite the strong correlation between NPO and IU, they may be
conceptually distinct constructs that are substantively linked, and as the recent models above suggest, NPO may be
an expression of IU. It is beyond the scope of the present review to establish which of these possibilities appears to
be the most plausible, but this is an important topic for future research.

Limitations
The review summarized and synthesised the findings of correlational research. Therefore, causality between NPO
and worry cannot be inferred. Further research, including experimental manipulations or longitudinal designs that

1
No differences were identified between studies that used the SPSI (considers NPO to exist on same scale as positive problem
orientation [PPO]) and the SPSI-R (considers NPO to exist on a separate sale to PPO).
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 335

can show temporal precedence, is required to clarify causality between NPO and worry. The present reviewed
included 33 effect sizes. While a large Orwin’s fail safe N indicated that the review’s findings were robust, a larger
number of effect sizes would have allowed potential moderators to be examined in more detail. A consideration for
future research may be to use more relaxed inclusion criteria and include more grey literature. In some of the
moderator analyses the meta-analytic assumption of independence was broken. While this is a weakness of the
current study, it represents a trade-off as this approach increased the power of the study by maximizing the number
of studies included. In addition, effect sizes from the same study were assigned half weights to ensure their
contribution was not overemphasised. Only seven out of the 33 effect sizes used in the study were from clinical
samples. Therefore, the field and the review somewhat neglects clinical populations and instead mainly assessed
the role of NPO in worry within analogue samples. Future research should concentrate on examining NPO within
clinical samples. It is also possible that rather than the review examining differences in the relationship between
somewhat theoretically different subgroups of NPO measures and worry, the review may have simply examined
whether NPO measures have increased in construct validity. Nevertheless, the subsets appeared to have good face
validity and were found to have significantly different relationships to worry. While the present review has begun to
investigate the NPO construct in detail, further research is required to establish the key constructs that underpin
NPO.

Future Directions
The current findings have important implications for future research and clinical practice. On the assumption that
NPO has some causal or maintaining role in worry, the first recommendation is for future worry interventions to
continue to target NPO. Future interventions should 1) continue to target NPO rather than problem solving skill per
se and 2) examine individual’s cognitive and emotional appraisals of problem solving scenarios, rather than simply
their ‘problem solving confidence’. Nevertheless, it remains unclear whether such interventions would be targeting
NPO, or instead a facet of IU. Therefore, an important goal for future research is to disentangle and help clarify the
relationship between these two constructs.

Conclusion
Despite NPO receiving considerable interest in the literature, in a recent review of prominent GAD models, it only
featured in one (Behar et al., 2009). It was therefore timely to review the relationship between NPO and worry. A
random effects model was used to synthesise the findings of 33 effect sizes, and 5,376 participants, to show that
NPO was strongly correlated with worry. Significant heterogeneity was identified between the studies. The strength
of the relationship between NPO and worry was dependent on the type of NPO measure used. The review indicated
that recent NPO measures, which define it to be a belief set that negatively predisposes individuals towards problem
solving situations, were most strongly correlated to worry. However, it remains unclear whether NPO should be
considered as a cognitive, or a cognitive-emotional, construct, and the extent to which NPO and IU are independent
constructs. Clarification of these matters will help improve future theoretical models, which may ultimately lead to
more effective treatment of excessive worry and disorders such as GAD.

References
APA. (1994). Diagnostic and Statistical Manual of Mental Disorders (4th Edition). Washington, DC: American
Psychological Association.
Anderson, R. J. & Goddard, L. (2009). Social Problem-Solving Processes and Mood in College Students: An
Examination of Self-report and Performance-based Approaches. Cognitive Therapy Research, 33, 175-186.
https://doi.org/10.1007/s10608-007-9169-3
Barahmand, U. (2008). Age and gender differences in adolescenborkt worry. Personality and Individual
Differences, 45(8), 778-783. https://doi.org/10.1016/j.paid.2008.08.006
Behar, E., DiMarco, I. D., Hekler, E. B., Mohlman, J., & Staples, A. M. (2009). Current theoretical models of
generalized anxiety disorder (GAD): Conceptual review and treatment implications. Journal of Anxiety
Disorders, 23, 1011-1023. https://doi.org/10.1016/j.janxdis.2009.07.006
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 336

Belzer, K. D., D'Zurilla, T. J., & Maydeu-Olivares, A. (2002). Social problem solving and trait anxiety as predictors
of worry in a college student population. Personality and Individual Differences, 33(4), 573-585.
https://doi.org/10.1016/S0191-8869(01)00173-8
Birrell, J., Meares, K., Wilkinson, A., & Freeston, M. (2011). Toward a definition of intolerance of uncertainty: A
review of factor analytical studies of the intolerance of Uncertainty Scale. Clinical Psychology Review, 31(7),
1198-1208. https://doi.org/10.1016/j.cpr.2011.07.009
Borenstein, M., Hedher L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. West
Sussex, UK: John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470743386
Borkovec, T. D. (1985). Worry: A potentially valuable concept. Behaviour Research and Therapy, 23(4), 481-482.
https://doi.org/10.1016/0005-7967(85)90178-0
Borkovec, T. D., Alcaine, O. M., & Behar, E. (2004). Avoidance theory of worry and generalized anxiety disorder.
In: R. Heimberg, C. Turk, & D. Mennin (Eds.), Generalized anxiety disorder: Advances in Research and
Practice. New York: Guilford Press. https://doi.org/10.1016/0005-7967(83)90121-3
Borkovec, T., Robinson, E., Pruzinsky, T., & Dupree, J. (1983). Preliminary exploration of worry: some
characteristics and processes. Behaviour Research and Therapy, 21, 9-16.
Bottesi, G., Heary, S. Ham, H., Peden, R. & Freeston, M. H. (2012). The intolerance of uncertainty model revisited:
How do the model components relate to each other? Paper presented at the 42nd annual EABCT Congress,
Geneva, Switzerland.
Carleton, N. R., Gosselin, P., & Asmundson, G. J. G. (2010). The Intolerance of Uncertainty Index: Replication and
Extension with an English Sample. Psychological Assessment, 22(2), 396-406.
https://doi.org/10.1037/a0019230
Carleton, N. R. (2012). The intolerance of uncertainty construct in the context of anxiety disorders: theoretical and
practical perspectives. Expert Review of Neurotherapeutics, 12(8), 937-947. https://doi.org/10.1586/ern.12.82
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. New Jersey, NJ: Lawrence Publishers.
Coutu, M. F., Durand, M. J., Marchand. A., Labrecque, M. E., Berbiche, D., & Cadieux, G. (2013). Factors
associated with generalized anxiety in workers undergoing rehabilitation for persistent musculoskeletal pain.
Disability and Rehabilitation, 35(19), 1599-1607. https://doi.org/10.3109/09638288.2012.748833
D’Zurilla, T. J., & Nezu, A. M. (1990). Development and preliminary evaluation of the Problem Solving Inventory.
Psychological Assessment, 2, 156-163. https://doi.org/10.1037/1040-3590.2.2.156
D’Zurilla, T. J., & Sheedy, C. F. (1991). Relationship between social problem-solving ability and subsequent level of
psychological stress in college students. Journal of Personality and Social Psychology, 61(5), 841-846.
https://doi.org/10.1037/0022-3514.61.5.841
D’Zurilla, T. J., Maydeu-Olivares, A., & Kant, G. L. (1998).Age and gender differences in social problem-solving
ability. Personality and Individual Differences, 25(2), 241-252. https://doi.org/10.1016/S0191-8869(98)00029-4
D'Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (1998). Manual for the social problem-solving inventory-revised.
North Yonawanda New York: Multi-health Systems.
Davey, G. C., Hampton, J., Farrell, J., & Davidson, S. (1992). Some characteristics of worrying: Evidence for
worrying and anxiety as separate constructs. Personality and Individual Differences, 13(2), 133-147.
https://doi.org/10.1016/0191-8869(92)90036-O
Davey, G. (1994). Worrying, social problem-solving abilities, and social problem-solving confidence. Behaviour
Research and Therapy, 32(3), 327-330. https://doi.org/10.1016/0005-7967(94)90130-9
Davey, G., Jubb, M., & Cameron, C. (1996). Catastrophic worrying as a function of changes in problem-solving
confidence. Cognitive Therapy and Research, 20(4), 333-344. https://doi.org/10.1007/BF02228037
Davey, G. C., & Levy, S. (1999). Internal statements associated with catastrophic worrying. Personality and
Individual Differences, 26(1), 21-32. https://doi.org/10.1016/S0191-8869(98)00162-7
De Vlieger, P. P., Crombez, G. G., & Eccleston, C. C. (2006). Worrying about chronic pain. An examination of
worry and problem solving in adults who identify as chronic pain sufferers. Pain, 120(1-2), 138-144.
https://doi.org/10.1016/j.pain.2005.10.022
Drews, A., & Hazlett-Stevens, H. (2008). Relationships between irritable bowel syndrome, generalized anxiety
disorder, and worry-related constructs. International Journal of Clinical and Health Psychology, 8(2), 429-436.
Dugas, M., Letarte, H., Rheaume, J., Freeston, M., & Ladouceur, R. (1995). Worry and problem solving: Evidence
of a specific relationship. Cognitive Therapy and Research, 19, 109-120. https://doi.org/10.1007/BF02229679
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 337

Dugas, M. J. (1997). Intolerance of uncertainty and problem orientation in worry. Cognitive Therapy and Research,
21(6), 593-606. https://doi.org/10.1023/A:1021890322153
Dugas, M., Gagnon, F., Ladouceur, R., & Freeston, M. (1998). Generalized anxiety disorder: A preliminary test of a
conceptual model. Behaviour Research and Therapy, 36(2), 215-226. https://doi.org/10.1016/S0005-
7967(97)00070-3
Dugas, M., Marchand, A., & Ladouceur, R. (2005). Further validation of a cognitive-behavioral model of generalized
anxiety disorder: Diagnostic and symptom specificity. Journal of Anxiety Disorders, 19(3), 329-343.
https://doi.org/10.1016/j.janxdis.2004.02.002
Dugas, M., Savard, P., Gaudet, A., Turcotte, J., Laugesen, N., Robichaud, M., & Koerner, N. (2007). Can the
Components of a Cognitive Model Predict the Severity of Generalized Anxiety Disorder? Behavior Therapy,
38(2), 169-178. https://doi.org/10.1016/j.beth.2006.07.002
Dupuy, J. B., & Beaudoin, S. (2001). Worry: daily self-report in clinical and non-clinical populations. Behaviour
Research and Therapy, 39(10), 1249-1255. https://doi.org/10.1016/S0005-7967(01)00011-0
Dupuy, J. B., & Ladouceur, R. (2008). Cognitive processes of generalized anxiety disorder in comorbid generalized
anxiety disorder and major depressive disorder. Journal of Anxiety Disorders, 22, 505-514.
https://doi.org/10.1016/j.janxdis.2007.05.010
Elliott, T., R., Sherwin, E., Harkins, S., W., & Marmarosh, C. (1995).Self-appraised problem solving ability, affective
states, and psychological distress. Journal of Counselling Psychology, 42, 105-115.
https://doi.org/10.1037/0022-0167.42.1.105
Fergus, T., & Wu, K. (2010). Do symptoms of generalized anxiety and obsessive-compulsive disorder share
cognitive processes? Cognitive Therapy and Research, 34(2), 168-176. https://doi.org/10.1007/s10608-009-
9239-9
Fergus, T. A., & Wu, K. D. (2011). Searching for specificity between cognitive vulnerabilities and mood and anxiety
symptoms. Journal of Psychopathology and Behavioral Assessment, 33(4), 446-458.
https://doi.org/10.1007/s10862-011-9245-6
Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry?
Personality and Individual Differences, 17(6), 791-802. https://doi.org/10.1016/0191-8869(94)90048-5
Gentes, E. L., & Ruscio, A. M. (2011). A meta-analysis of the relation of intolerance of uncertainty to symptoms of
generalized anxiety disorder, major depressive disorder, and obsessive-compulsive disorder. Clinical
Psychology Review, 31, 923-933. https://doi.org/10.1016/j.cpr.2011.05.001
Gonzalez Rodriguez, M., Rovella, A., Barbenza, C., & Rausch, L. (2011). Trait worry and generalized anxiety
disorders in adolescents: cognitive processes contributions. Revista Iberoamericana De Diagnostico Y
Evaluacion-E Avaliacao Psicologica, 2(32), 123-142.
Gosselin, P., Dugas, M., & Ladouceur, R. (2002). Worry and problem solving: The role of negative problem
orientation. Journal de Therapie Comportementale et Cognitive, 12(2), 49-58.
Gosselin, P., Ladouceur, R., & Pelletier, O. (2005). Evaluation of an individual's attitude toward daily life problems:
The Negative Problem Orientation Questionnaire. Journal de Therapie Comportementaleet Cognitive, 15(4),
141-153. https://doi.org/10.1016/S1155-1704(05)81235-2
Hedges, L. V., & Olkin, I. (1985).Statistical merthods for meta-analysis. Orlando Florida: Academic Press.
Heppner, P. P., & Petersen, S. (1982). The development and implications of a personal problem-solving inventory.
Journal of Counselling Psychology, 29, 66. https://doi.org/10.1037/0022-0167.29.1.66
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D., G. (2003). Measuring inconsistency in meta-analysis.
British Medical Journal, 327, 557-560. https://doi.org/10.1136/bmj.327.7414.557
Hutcutt, A. I., & Arthur, W. (1995). Development of a new outlier statistic for meta-analytic data. Journal of
Consulting and Clinical Psychology, 74, 401-415.
Kertz, S., & Woodruff-Borden, J. (2013). The role of metacognition, intolerance of uncertainty, and negative
problem orientation in children's worry. Behavioural and Cognitive Psychotherapy, 41(2), 243-248.
https://doi.org/10.1017/S1352465812000641
Khawaja, N. G., & Chapman, D. (2007). Cognitive predictors of worry in a non-clinical population. Clinical
Psychologist, 11(1), 24-32. https://doi.org/10.1080/13284200601151778
Koerner, N., Dugas, M. J., Gosselin, P., & Langlois, F. (2013). A Preliminary Test of the Intolerance of Uncertainty
Model of Generalized Anxiety Disorder. Paper presented at the CACBT 2013 Conference, Montreal, Canada.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 338

Ladouceur, R., Blais, F., Freeston, M. H., & Dugas, M. J. (1998). Problem solving and problem orientation in
generalized anxiety disorder. Journal of Anxiety Disorders, 12(2), 139-152. https://doi.org/10.1016/S0887-
6185(98)00002-4
Ladouceur, R., Dugas, M. J., Freeston, M. H., Rheaume, J., Blais, F., Boisvert, J.-M., & Thibodeau, N. (1999).
Specificity of Generalized Anxiety Disorder Symptoms and Processes. Behavior Therapy, 30(2), 191-207.
https://doi.org/10.1016/S0005-7894(99)80003-3
Ladouceur, R., Gosselin, P., & Dugas, M. J. (2000). Experimental manipulation of intolerance of uncertainty: a
study of theoretical model of worry. Behaviour Research and Therapy, 38, 933-941.
https://doi.org/10.1016/S0005-7967(99)00133-3
Laugesen, N., Dugas, M. J., Bukowski, W. M (2003). Understanding Adolescent Worry: The Application of a
Cognitive Model. Journal of Abnormal Child Psychology, 31(1), 55-64. https://doi.org/10.1023/A:1021721332181
Lewinsohn, P. M., Gotlib, I. H., Lewinsohn, M. A., Seeley, J. R., & Allen, N. B. (1998). Gender differences in anxiety
disorders and anxiety symptoms. Journal of Abnormal Psychology, 107, 109-117. https://doi.org/10.1037/0021-
843X.107.1.109
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications, Inc.
Matthews, A. (1990). Why worry? The cognitive function of anxiety. Behaviour Research and Therapy, 28, 455-
468. https://doi.org/10.1016/0005-7967(90)90132-3
Maydeu-Olivares, A., & D’Zurilla, T. J. (1996). A factor-analytic study of the Social Problem-Solving Inventory: An
integration of theory and data. Cognitive Therapy and Research, 20, 115-133.
https://doi.org/10.1007/BF02228030
McManus, S., Meltzer, H., & Brugha, T. (2009). Adult Psychiatric Morbidity in England, 2007: Results of a
Household Survey. Leeds.
Meeten, F., Dash, S. R., Scarlet, A. L. S., & Davey, G. C. L. (2012). Investigating the effect of intolerance of
uncertainty on catastrophic worrying and mood. Behaviour Research and Therapy, 50, 690-698.
https://doi.org/10.1016/j.brat.2012.08.003
Mennin, D. S., Heimberg, R. G., Turk, C. L., & Fresco, D. M. (2005). Preliminary evidence for an emotion
dysregulation model of generalized anxiety disorder. Behaviour Research and Therapy, 43, 1281-1310.
https://doi.org/10.1016/j.brat.2004.08.008
Nezu, M. (1986). Negative life stress and anxiety – problem-solving as a moderator variable. Psychological
Reports, 58(1), 279-283. https://doi.org/10.2466/pr0.1986.58.1.279
Nezu, M. (2004). Problem Solving and Behavior Therapy Revisited. Behavior Therapy, 35, 1-33.
https://doi.org/10.1016/S0005-7894(04)80002-9
Parkinson, M., & Creswell, C. (2011). Worry and problem-solving skills and beliefs in primary school children.
British Journal of Clinical Psychology, 50(1), 106-112. https://doi.org/10.1348/014466510X523887
Pretorius, T. B. (1992).Problem-solving appraisal in the association of life stress and depression – a South-African
study. Psychological Reports, 71(3), 855-862. https://doi.org/10.2466/pr0.1992.71.3.855
Rausch, L., Rovella, A., de Barbenza, C. M., & Rodriguez, M. G. (2011). Cognitive processes of the generalized
anxiety disorder in adolescents. Psicologia y Salud, 21(2), 215-226.
Robichaud, M., Dugas, M., & Conway, M. (2003). Gender differences in worry and associated cognitive-behavioral
variables. Journal of Anxiety Disorders, 17(5), 501-516. https://doi.org/10.1016/S0887-6185(02)00237-2
Robichaud, M., & Dugas, M. J. (2005). Negative problem orientation (Part 1): psychometric properties of a new
measure. Behaviour Research and Therapy, 43(3), 391-401. https://doi.org/10.1016/j.brat.2004.02.007
Robichaud, M., & Dugas, M. J. (2005). Negative problem orientation (Part II): construct validity and specificity to
worry. Behaviour Research and Therapy, 43, 403-412. https://doi.org/10.1016/j.brat.2004.02.008
Roemer, L., & Orsillo, S. M. (2002). Expanding our conceptualization of and treatment for generalized anxiety
disorder: integrating mindfulness/acceptance-based approaches with existing cognitive behavioural models.
Clinical Psychology: Science and Practice, 9, 54-68. https://doi.org/10.1093/clipsy.9.1.54
Rosental, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
https://doi.org/10.4135/9781412984997
Rovella, A., Gonzalez, M., Penate, W., & Ibanez, I. (2011). Worry trait and generalised anxiety disorder in a sample
of general population: The differential role of intolerance of uncertainty, cognitive avoidance, negative problem
orientation, and meta-beliefs. Revista Argentina De Clinica Psicologica, 20(2), 101-108.
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 339

Ruscio, A. M., & Seitchik, A. E. (2007, November). Problem-solving abilities and appraisals associated with severe
worry and GAD. In A. M. Ruscio (Chair), Cognitive vulnerabilities in generalized anxiety disorder: Basic research
findings and implications for practice. Symposium presented at the annual meeting of the Association for
Behavioral and Cognitive Therapies, Philadelphia, PA.
Smith, K. (2012). Negative Problem Orientation, Cognitive Avoidance and Worry among Young Adolescents
(Unpublished Undergraduate Thesis). Newcastle University School of Psychology. Newcastle Upon Tyne.
Spielberger, C. D. (2010). State-Trait Anxiety Inventory. Corsini Encyclopedia of Psychology. South Florida: John
Wiley & Sons.
Sugiura, Y. (1999). Worry: Problem-solving orientation and uncontrollability. Japanese Journal of Educational
Psychology, 47(2), 191-198. https://doi.org/10.5926/jjep1953.47.2_191
Sugiura, Y. (2002). Problem-focused coping strategies and the uncontrollability of thoughts: Mediating role of meta-
cognition. Japanese Journal of Educational Psychology, 50(3), 271-282.
https://doi.org/10.5926/jjep1953.50.3_271
Sugiura, Y. (2004). Detached mindfulness and worry: a meta-cognitive analysis. Personality and Individual
Differences, 37(1), 169-179. https://doi.org/10.1016/j.paid.2003.08.009
Sugiura, Y. (2007). Responsibility to continue thinking and worrying: evidence of incremental validity. Behaviour
Research and Therapy, 45(7), 1619-1628. https://doi.org/10.1016/j.brat.2006.08.001
Szabo, M., & Lovibond, P. F. (2002). The Cognitive Content of Naturally Occurring Worry Episodes. Cognitive
Therapy and Research, 26(2), 167-177. https://doi.org/10.1023/A:1014565602111
Sazbo, M., & Lovibond, P. F. (2004). The Cognitive Content of Thought-Listed Worry Episodes in Clinic-Referred
Anxious and Nonreferred Children. Journal of Clinical Child and Adolescent Psychology, 33(3), 613-622.
https://doi.org/10.1207/s15374424jccp3303_18
Szabo, M., & Lovibond, P. F. (2006). Worry episodes and perceived problem solving: a diary-based approach.
Anxiety, Stress, and Coping, 19(2), 175-187. https://doi.org/10.1080/10615800600643562
Thompson, T., Webber, K., Montgomery, I. (2002). Performance and persistence of worriers and non-worriers
following success and failure feedback. Personality and Individual Differences, 33, 837-848.
https://doi.org/10.1016/S0191-8869(01)00076-9
Wells, A. (1995). Meta-cognition and worry: a cognitive model of generalized anxiety disorder. Behavioural and
Cognitive Psychotherapy, 23, 301-320. https://doi.org/10.1017/S1352465800015897
Wilson, C. C., & Hughes, C. C. (2011). Worry, beliefs about worry and problem solving in young children.
Behavioural and Cognitive Psychotherapy, 39(5), 507-521. https://doi.org/10.1017/S1352465811000269
Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand, 67, 361-
370. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x
Psychopathology Review, Volume 4 (2017), Issue 3, 319-340 340

Appendices
Appendix 1: Comparison of NPO-Q and IUS Items

NPO-Q IUS
Similar Items
I see problems as a danger I must get away from all things I am unsure of (I-
IU)
I think that problems will get in the way of things I Feeling unsure stops me from doing most things
need to do
When I have a problem, the first think I do is think When I’m not sure what to do I freeze
that I do not have the ability to solve it
I see problems as a threat to my well-being When I don’t know what will happen, I can’t do
things very well
When I think about all the ways to solve a The smallest worry can stop me from doing things
problem, I still wonder if my answer will work
It often seems like I can’t beat my problems Even if you plan things really well, one little thing
can ruin it
Remaining items

I think my problems as bigger than they really are When things happen suddenly, I get very upset
I often doubt whether I can solve problems It bothers me when there are things I don’t know
Before I try to solve a problem, I think that it is People should always think about what will
hard to solve problems happen next. This will stop bad things from
happening
When I try to solve a problem I often think I can’t I always want to know what will happen to me in
the future
I often feel that my problems cannot be solved I can’t stand it when things happen suddenly
I think my problems will not go away, even if I can I should always be prepared before things happen
think of ways to solve them

You might also like