Cognitive Behaviour Therapy

ISSN: 1650-6073 (Print) 1651-2316 (Online) Journal homepage: https://www.tandfonline.com/loi/sbeh20

Internet-delivered cognitive behavioral therapy

for panic disorder with or without agoraphobia: a
systematic review and meta-analysis

Eileen P. Stech, Jaclyn Lim, Emily L. Upton & Jill M. Newby

To cite this article: Eileen P. Stech, Jaclyn Lim, Emily L. Upton & Jill M. Newby (2019): Internet-
delivered cognitive behavioral therapy for panic disorder with or without agoraphobia: a systematic
review and meta-analysis, Cognitive Behaviour Therapy, DOI: 10.1080/16506073.2019.1628808

To link to this article: https://doi.org/10.1080/16506073.2019.1628808

View supplementary material

Published online: 15 Jul 2019.

Submit your article to this journal

Article views: 14

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=sbeh20
COGNITIVE BEHAVIOUR THERAPY
https://doi.org/10.1080/16506073.2019.1628808

Internet-delivered cognitive behavioral therapy for panic

disorder with or without agoraphobia: a systematic review
and meta-analysis
Eileen P. Stech , Jaclyn Lim , Emily L. Upton and Jill M. Newby
School of Psychology, University of New South Wales, Sydney, Australia

ABSTRACT ARTICLE HISTORY

The current systematic review and meta-analysis examined the effi- Received 18 December 2018
cacy and effectiveness of internet-delivered cognitive behavioral Accepted 4 June 2019
therapy (iCBT) on panic disorder and agoraphobia symptom severity. KEYWORDS
Twenty-seven studies were identified. Results from nine randomised Internet-delivered cognitive
controlled trials (RCTs) showed that iCBT outperformed waiting list behavioral therapy; panic
and information controls for panic (g = 1.22) and agoraphobia disorder; agoraphobia;
(g = .91) symptoms, but the quality of RCTs varied and heterogeneity meta-analysis; systematic
was high. Results from three RCTs suggested iCBT may have similar review
outcomes to face-to-face CBT in reducing panic and agoraphobia
symptoms. Within-group effect sizes between baseline and post-
treatment were large for panic (n = 29, g = 1.16) and medium for
agoraphobia symptom severity (n = 18, g = .73). Subgroup analyses of
within-group pre/post treatment effect sizes showed larger within-
group effect sizes for efficacy studies (n = 15) compared to effective-
ness studies (n = 14) for panic severity (g = 1.38 vs. g = .98) but not
agoraphobia severity. There was no impact of program length, inclu-
sion or arousal reduction techniques, or degree of clinician support.
Within-group effects of iCBT suggest the reduction in panic and
agoraphobia symptom severity is maintained at 3–6 month follow-
up (n = 12).

Panic disorder is characterized by a persistent concern about future panic attacks, catastrophic
cognitions about panic, and significant changes in behavior to avoid future attacks (American
Psychiatric Association, 2013). Between 35–65% of individuals with panic disorder also meet
criteria for agoraphobia: intense fear and avoidance of situations in which it is perceived that
escaping or getting help may be difficult if incapacitating or panic-like symptoms develop
(APA, 2013; Wittchen, Gloster, Beesdo-Baum, Fava, & Craske, 2010). Individuals diagnosed
with panic disorder experience a significantly lower quality of life than non-anxious indivi-
duals (Barrera & Norton, 2009), with notable impairment in social functioning and engage-
ment in work and family responsibilities (Sherbourne, Wells, & Judd, 1996). The presence of
comorbid agoraphobia is associated with increased impairment, and the severity of a person’s
agoraphobic avoidance is predictive of their degree of disability (Wittchen et al., 2010).

CONTACT Eileen P. Stech eileen.stech@unsw.edu.au School of Psychology, UNSW Sydney, Kensington, NSW
2052, Australia
Systematic Review Registration Number: PROSPERO: CRD42017069878.
Supplementary data for this article can be accessed here
© 2019 Swedish Association for Behaviour Therapy
2 E. P. STECH ET AL.

Cognitive behavioral therapy (CBT) is the most empirically supported psychological

treatment for panic disorder with or without agoraphobia (Pompoli et al., 2016). Clinical
guidelines published by the American Psychiatric Association (APA, 2009) and the
National Institute for Clinical Excellence (NICE, 2011) recommend CBT as a first-line
intervention for panic disorder. Despite this strong empirical base, CBT has long been
underutilized by clinicians in the community (Goisman, Warshaw, & Keller, 1999; Stein
et al., 2011; Wolitzky-Taylor, Zimmermann, Arch, De Guzman, & Lagomasino, 2015).
Contributing factors to the low provision of CBT for anxiety include lack of clinician
training and competence, clinician biases, organisational barriers, and logistical barriers at
a patient level (Wolitzky-Taylor et al., 2018).
Delivering CBT via automated internet programs is one effective method of
increasing dissemination of empirically supported interventions with fidelity
(Andrews, Newby, & Williams, 2015). Internet-delivered CBT (iCBT) also alleviates
issues of accessibility due to lack of skilled practitioners in rural areas, prohibitive
costs, clients’ limited mobility or inflexible work schedules, and long waiting lists
(Andrews et al., 2015). iCBT for panic disorder resembles an online course and
typically entails 5–10 modules delivered across an 8–12 week period. Clients are
assigned skills practice between modules and may receive remote support from
a therapist via phone or email.
Several reviews and meta-analyses have evaluated the efficacy of iCBT for anxiety
disorders broadly and reported iCBT for anxiety disorders achieves large effect sizes
(e.g., Olthuis, Watt, Bailey, Hayden, & Stewart, 2015). For panic disorder specifically,
Olthuis et al. (2015) found clinician-guided iCBT had a large mean between-group
effect (1.52) on panic severity when compared to waitlist or control conditions.
However they did not examine impact on agoraphobic severity, and open and non-
randomized studies were excluded. This meant that most effectiveness studies con-
ducted in usual/routine care settings were excluded as randomization is often not
feasible in these settings. Outcomes of the included studies varied substantially (i.e.
heterogeneity was high), so further research is needed to explore what aspects of the
interventions or study design may account for the variation. In a follow up meta-
analysis, Andrews et al. (2018) reviewed the literature on iCBT for panic, but
restricted the sample to studies with a care as usual or inactive control condition,
and found large mean between-group effect size (1.31) for panic symptoms. They did
not include open trials and did not examine the influence of treatment or delivery
factors on outcomes, nor assess agoraphobic severity. Finally, in the only meta-
analysis to compare iCBT to face-to-face CBT, Carlbring, Andersson, Cuijpers,
Riper, and Hedman-Lagerlöf (2018) found equivalent efficacy between the two mod-
alities for panic symptoms; however, they did not examine measures of agoraphobia,
so were unable to comment on whether iCBT is similarly efficacious to face-to-face
CBT for reducing agoraphobia symptoms.
Each of the aforementioned meta-analyses has explored the efficacy of iCBT across
various anxiety disorders, and therefore contained limited information specific to iCBT
for panic disorder. However, the existing iCBT programs differ in length, degree of
clinician support, and the cognitive-behavioral techniques they include. Consequently,
it is unknown what features and components of iCBT programs are necessary or most
effective. For example, meta-analyses of face-to-face therapy suggest that exposure alone
 COGNITIVE BEHAVIOUR THERAPY 3

has equivalent efficacy to multi-component CBT (e.g., Norton & Price, 2007), and
a component network meta-analysis suggested that interoceptive exposure specifically
was a very potent component of CBT (Pompoli et al., 2018). In contrast, arousal
reduction via breathing retraining or progressive muscle relaxation does not increase
the efficacy of cognitive and exposure techniques and may lead to poorer long-term
outcomes (Pompoli et al., 2018; Schmidt et al., 2000). Within the iCBT literature, some
programs include arousal reduction techniques, while others place a greater emphasis
on cognitive and exposure techniques. Furthermore, exposure is often introduced in the
latter half of iCBT programs, meaning participants who drop out early may never be
introduced to exposure strategies.
Past reviews have also excluded open trials or non-randomized trials, which precluded
examination of the effectiveness of iCBT for panic disorder in usual care settings, since
evaluations of iCBT in these contexts do not often include control groups. An increasing
number of effectiveness studies of iCBT for panic disorder have been reported over the past
10 years, making a summary of these trials and comparison with efficacy trials a timely
endeavor. Additionally, no previous review has examined the effect of iCBT on agoraphobic
avoidance, despite its well-documented relationship to panic symptoms and impact on
functioning. Previous reviews have also focused on post-treatment outcome, with limited
consideration of the longer-term effects of iCBT for panic disorder. Finally, the search for
articles in the most recent meta-analysis was conducted over two years ago. The authors are
aware of at least three important studies that have been published since and are yet to be
included in any meta-analysis, including a large effectiveness study (Nordgreen, Gjestad,
Andersson, Carlbring, & Havik, 2018), a comparison of disorder-specific versus transdiag-
nostic iCBT (Fogliati et al., 2016), and a high quality RCT comparing iCBT (with varying
degrees of support) to a waitlist control (Ciuca, Berger, Crişan, & Miclea, 2018). Therefore,
an updated, detailed and inclusive systematic review and meta-analysis is needed to
examine the current state of the field and inform directions for future research.

Aims/objectives
The aim of this review was to provide a comprehensive and updated summary of the
published studies evaluating iCBT for symptoms of panic disorder and/or agoraphobia.
Our specific aims were: 1) to examine the efficacy of iCBT compared to control
conditions (waiting list control or information control) and face-to-face CBT on
panic and agoraphobia symptoms; 2) to examine the effects of iCBT on panic and
agoraphobia symptoms from pre- to post-treatment, and pre-treatment to follow-up; 3)
to describe the components included in various iCBT programs for panic disorder, and
estimate the proportion of participants who are introduced to exposure; and 4) to
investigate the impact of potential moderators of treatment effect including study
context (efficacy versus effectiveness), program length, degree of therapist support
and inclusion of arousal reduction techniques. We hypothesized that studies conducted
in efficacy settings and those with higher degree of therapist support would have larger
effects. Because the analyses examining the impact of program length and inclusion of
arousal reduction techniques were exploratory, no hypotheses were made. Finally, we
aimed to provide recommendations for future research and refinement of iCBT pro-
grams for panic and agoraphobia.
4 E. P. STECH ET AL.

Methods
Protocol and registration
We developed the protocol for this review in accordance with the procedures outlined
in the Cochrane Handbook for systematic reviews (Higgins & Green, 2011), and
registered the protocol with PROSPERO1 [CRD42017069878]. We followed the
PRISMA guidelines for the reporting of this systematic review and meta-analysis
(Moher, Liberati, Tetzlaff, & Altman, 2009).

Inclusion/eligibility criteria (PICOS)

Participants
We included studies with adults over the age of 18, with elevated symptoms of panic
disorder (according to validated self-report or clinician-rated scales) and/or a diagnosis
of panic disorder based on the Diagnostic and Statistical Manual of Mental Disorders
(DSM) or International Classification of Diseases (ICD) criteria.

Interventions
We included studies of an iCBT program specifically targeting panic disorder or panic
symptoms. Internet-delivered CBT was defined as being delivered online, remotely, and
without any face-to-face contact with a therapist during the intervention. Both self-help
and clinician-guided interventions were included.

Comparison
We included studies that reported comparisons between iCBT and waiting list control
(WLC), information control, treatment as usual or usual care groups, psychological
treatment placebo, face-to-face CBT, or alternative internet-delivered or face-to-face psy-
chological therapies in the systematic review. Open trials were also included. Studies that
reported comparisons between iCBT and WLC or information controls, as well as iCBT
versus face-to-face CBT, were included in the meta-analyses of between-groups effects.

Outcomes
We included studies that reported at least one validated self-report and/or clinician-rated
instrument of panic disorder or agoraphobia symptom severity before and after treatment,
meaning at minimum we were able to calculate the within-group effect size of iCBT on
primary outcome measures (and controlled effect sizes where iCBT was compared to
a control condition). To explore the maintenance of treatment gains, we extracted data at
3–6 months follow-up and 1–2 years follow-up in addition to post-treatment.

Study design
We included studies published in English in peer-reviewed journals between
January 1990 and 14 November 2018. We included randomized controlled trials, non-
randomized trials (i.e., studies that included a comparison group but allocation to
group was not random), and open trials with no comparison group. We included
studies in both efficacy and effectiveness (i.e., routine care) settings.
 COGNITIVE BEHAVIOUR THERAPY 5

Excluded studies
We excluded studies in which the intervention of interest was transdiagnostic or third-
wave (e.g., internet-delivered mindfulness or acceptance and commitment therapy). We
also excluded studies in which the intervention was delivered purely via smartphone
application. Case series and case reports were excluded.

Identification of studies
To identify studies that may fit the inclusion criteria, we conducted systematic searches of
the electronic databases PSYCInfo, EBM Reviews—Cochrane Central Register of
Controlled Trials, EMBASE, OVID MEDLINE(R) and PubMed up to
14 November 2018. See Appendix A for our electronic search strategy for PSYCInfo. To
optimize the sensitivity of our search, we combined terms indicative of panic disorder and
agoraphobia with terms expressive of cognitive or behavioral therapy and internet-
delivered format. Additionally, we searched the reference lists of eligible studies and
previous review articles to identify other possibly eligible studies. No attempts were made
to identify unpublished data relevant to the topic.

Study selection
The first and last author (ES and JN) independently screened the titles and abstracts of
all retrieved studies to determine their relevance and discarded studies that could be
excluded based on this information alone. ES and JN then independently assessed the
remaining references for eligibility by reviewing the full text of the articles, and
disagreements were resolved through discussion.

Data extraction and management

ES, JL and EU extracted data regarding the study design, intervention and outcome
measures into a Microsoft Excel spreadsheet. Data relevant to the meta-analysis was
transferred to Comprehensive Meta-Analysis (version 3.0; Biostat, Inc.). The following
information was extracted from each study: authors; year of publication; country where
study was conducted; mean age; % female; eligibility/diagnostic assessment measure;
proportion of participants with comorbid diagnoses; context; study design; type and
degree of clinician support; length (weeks), number of modules, and components of
iCBT program; type of comparison group; adherence; attrition; adverse events; and
outcome data from measures assessing panic and agoraphobia symptom severity. Data
was not double-coded during the extraction phase.

Primary and secondary outcomes

Primary outcomes
i) Between-group effect sizes comparing iCBT and waitlist or information controls on
panic disorder and agoraphobia symptom severity at post-treatment.
6 E. P. STECH ET AL.

ii) Between-group effect sizes comparing iCBT and face-to-face CBT on panic disorder
and agoraphobia symptom severity at post-treatment.

Secondary outcomes
i) Within-group effect sizes comparing panic disorder and agoraphobia symptom
severity from pre-treatment to post-treatment.
ii) Within-group effect sizes comparing panic and agoraphobia symptom severity from
pre-treatment to 3–6 months follow-up and pre-treatment to 1–2 years follow-up. If
two time points were reported within a specified follow-up period, then the later time
point was used.
iii) Adherence (mean number of lessons completed and percentage of participants who
completed the full program) and proportion of participants who were introduced to
exposure (by comparing adherence data to the module in which exposure was
introduced).
iv) Adverse reactions (participants with symptom worsening, or other negative effects
during the intervention or control period).
Across all outcomes, symptom severity was according to validated continuous symptom
measures. Clinician-rated and self-report instruments were analyzed together to max-
imize the number of studies included in each analysis. For studies that reported more
than one instrument to measure panic or agoraphobia symptoms, an instrument was
selected according to an a priori defined order based on the most frequently used
measures across trials (see Appendix B). Reported means and standard deviations were
used to calculate effect size directly; however, if not available, then the Cohen’s d and
95% confidence interval reported by the authors were used.

Risk of bias in individual studies

The quality and risk of bias of included studies was assessed by ES using the Risk of Bias
tool (Higgins & Green, 2011) for RCTs and a simplified version (i.e. risk classified as
low, high or unclear) of the ROBINS-I tool for non-randomized and open studies
(Sterne et al., 2016). Both tools were developed by the Cochrane Collaboration.

Statistical analyses
Calculation of controlled effect sizes: ICBT versus comparison at post-treatment
Standardized mean difference (effect size) scores and their 95% confidence intervals
were calculated for each comparison between iCBT and information or waitlist control
conditions, and between iCBT and face-to-face CBT, by subtracting mean scores and
dividing by the pooled standard deviation. For studies with multiple comparisons, we
combined the relevant intervention conditions to create one pair-wise comparison to
the control condition (Higgins & Green, 2011). To address the small sample sizes, we
adjusted the effect sizes according to the procedures described by Hedges and Olkin
(1985, or Hedges g). Effect sizes of 0.2, 0.5, and 0.8 were considered small, moderate
and large respectively (Cohen, 1988).
 COGNITIVE BEHAVIOUR THERAPY 7

Calculation of within-group effect sizes: changes in primary outcome measures

between pre- and post-treatment, and pre-treatment and follow-up
We calculated within-group effect sizes and their 95% confidence intervals by subtract-
ing the pre-treatment mean score on each primary outcome measure from the post-
treatment (or follow-up) mean score, and dividing by the pooled standard deviation.
Due to the small sample sizes we used Hedges g, which required the correlation between
values at each time point since the observed values were not independent of one
another. As most articles did not report the correlation between time points, we entered
a conservative value of 0.50 (Balk, Earley, Patel, Trikalinos, & Dahabreh, 2012).2
We used the Comprehensive Meta-Analysis (version 3.0, Biostat Inc.) software to
calculate pooled mean effect sizes. Given the substantial heterogeneity amongst the
study designs evidenced in the extraction process, we calculated the mean effect sizes
using a random effects model. The random effects model assumes there is variation in
the true effect size among studies, and that the included studies reflect a random sample
of the possible effect sizes. Therefore, the summary effect is an estimate of the average of
all effect sizes (Borenstein, Hedges, Higgins, & Rothstein, 2010).

Subgroup analyses
Subgroup analyses were conducted using the within groups pre- to post-treatment data,
as this data set included the largest number of studies. The variables of interest were
planned a priori: efficacy versus effectiveness studies, program length, degree of thera-
pist support, and inclusion of arousal reduction techniques. The parameters used to
classify the length of programs and degree of therapist support were defined post-hoc,
to ensure a sufficient number of studies in each grouping. A mixed-effects model was
used; whereby a random-effects model was used to pool studies within subgroups and
a fixed-effects model to test for significant differences between subgroups.

Testing homogeneity
We calculated the Q statistic and the I 2 statistic to test the consistency of effect sizes.
A significant Q statistic rejects the null hypothesis of homogeneity around effect sizes;
whereas the I 2 statistic indicates the degree of heterogeneity across effect sizes as
a percentage score, where a value of 0% indicates no heterogeneity. Scores of 25%, 50%
and 75% indicate low, moderate, and high heterogeneity, respectively.

Risk of bias across studies

Testing for and dealing with publication bias
We reviewed the funnel plots of the primary outcome measures of panic and agoraphobia
for asymmetry, which suggests there may be a small study effect that is potentially related
to publication bias (Egger, Davey Smith, Schneider, & Minder, 1997). We then used
Duval and Tweedie’s Trim and Fill procedure (Duval & Tweedie, 2000a, 2000b) to
calculate an adjusted effect size, which accounts for any observed publication bias to
produce a best estimate of the unbiased effect size.
8 E. P. STECH ET AL.

Results
Study selection
Figure 1 presents the flowchart describing the inclusion of studies. We examined
a total of 469 titles and abstracts, of which 343 were rejected. A further 47 articles
were rejected after the entire article was reviewed. Therefore, 27 studies (from 26
articles) were included: 15 RCTs, 10 open (uncontrolled) trials, and two non-
randomized trials. Nine of the RCTs compared iCBT to a WLC or information
control group.

807 records identified through

database searching
PSYCInfo, EBM Reviews - Cochrane
Central Register of Controlled Trial,
EMBASE, and OVID MEDLINE(R)
Identification

(n=649)
PubMed (n=158)
5 additional records identified
through other sources

469 records after duplicates (n=343) removed

Screening

469 records screened 396 records excluded based

on title and abstract

73 full-text articles assessed for 47 of full-text articles

eligibility excluded, with reasons

Duplicate/overlapping data
with another included study
Eligibility

(n=5)
27 studies (from 26 articles)
No outcome
included in qualitative
data/inappropriate data (e.g.,
synthesis
review, n=7)
n=15 RCTs
No validated outcome
n=10 open trials measure (n=3)
n=2 non-randomised trials Not internet delivered/not
delivered remotely (n=6)
Involved face-to-face
component (n=14)
Included

Prevention program (n=1)

Treatment (transdiagnostic)
Studies included in quantitative (n=7)
synthesis (meta-analysis):
Treatment (not CBT) (n=4)
n= 9 RCTs (studies with WLC
or IC only)
n=29 (from 27 studies; baseline
to post-treatment)
n=14 (from 12 studies; baseline
to 3-6 month follow-up)

Figure 1. Study flow chart.

 COGNITIVE BEHAVIOUR THERAPY 9

Study characteristics
Table 1 summarizes the characteristics of the included studies (see Appendix C for full
reference details of included studies). The sample sizes of groups within the studies
ranged from six to 570. Samples were predominately female (mean proportion = 68%)
and had a mean age of 37 years. The majority of studies were completed in Australia
(13 studies) or Sweden (8 studies). All but two studies (unguided condition in Ciuca
et al., 2018; Klein, Shandley, Austin, & Nordin, 2008) included clinician supported
iCBT; most via email only (n = 16), however seven studies included email and phone,
and three studies provided support via phone/video call only. The intervention periods
varied in length from 5 to 13 weeks, with the iCBT programs including between 5 and
17 modules or lessons.

Treatment components of iCBT programs

All of the iCBT programs included modules on psychoeducation and cognitive restructur-
ing, and most included a module devoted to relapse prevention. Most studies (n = 21,
78%) included arousal reduction techniques (controlled breathing or progressive muscle
relaxation), although five did not. All but one program (Richards & Alvarenga, 2002)
instructed participants in gradual exposure, usually to both bodily sensations (interocep-
tive exposure) and feared situations (in-vivo exposure). Most programs began with
cognitive restructuring or arousal reduction techniques, before proceeding to exposure.
Different research groups also incorporated various other components into their pro-
grams, such as assertiveness training or stress management modules.

Pre-treatment diagnostic assessment and outcome measures

The majority of studies (n = 23; 85%) required participants to meet DSM-IV criteria for
Panic Disorder, established by a structured clinical interview. Most studies employed the
Panic Disorder Severity Scale, Panic Disorder Severity Scale—Self-Report, or Body
Sensations Questionnaire to measure panic symptoms; and the 16 studies that assessed
agoraphobia symptoms mostly used the Mobility Inventory—Alone subscale or Clinician
Severity Rating from the Anxiety and Related Disorders Interview Schedule for DSM-5.

Risk of bias within randomized control trials

Many studies did not report a trial registration number or published protocol, which
precluded the determination of risk of selective reporting of results. All studies were
coded as low risk for handling of incomplete data, 12 (80%) reported adequate genera-
tion of random sequence, and 12 (80%) reported adequate concealment of allocation.
All studies were coded as high risk for blinding of participants and personnel and five
studies (33%) were rated as low risk for blinding of outcome assessment. Three studies
were classified as high risk overall, as more than three domains were rated as having
high or unclear risk (see Table 2).
 10

Table 1. Characteristics of studies included in the meta-analysis evaluating iCBT for panic disorder.
Mean % # of Length in Support type; Primary
Study Country Context Participants % with Ag age female Design Treatment arm: n modules weeks clinician type Measures
Efficacy studies
Allen 2016 Australia Efficacy DSM-IV PD Not stated 38 84 RCT iCBT: 30 5 8 Email, phone; PD: PDSS-SR
Study 1 (MINI) WLC: 37 therapist,
psychiatrist
Carlbring Sweden Efficacy DSM-IV PD Not stated 34 71 RCT iCBT: 21 6 7-12 Email; therapist PD: BSQ
E. P. STECH ET AL.

2001 (ADIS-IV) WLC: 20 Ag: MI-A

Carlbring Sweden Efficacy DSM-IV PD 91 38 68 RCT iCBT: 11 6 Within 6 Email; therapist PD: BSQ
2003 (SCID) Internet- months Ag: MI-A
delivered
applied
relaxation: 11
Carlbring Sweden Efficacy DSM-IV PD 51 35 71 RCT iCBT: 25 10 10 Email; clinical PD: BSQ
2005 (SCID) Face to face psychologists, Ag: MI-A
CBT: 24 trainee
psychologists
Carlbring Sweden Efficacy DSM-IV PD Not stated 37 60 RCT iCBT: 30 10 10 Email, phone; PD: BSQ
2006 (SCID & WLC: 30 psychologist, Ag: MI-A
ADIS-IV) trainee
psychologists
Ciuca Romania Efficacy DSM-IV PD 52 35 68 RCT iCBT-unguided: 37 17 12 None PD: PDSS-SR
2018* (PDSQ iCBT-guided: 36 or
based Waitlist: 38 Video call
interview) (Skype); clinical
psychologists
Fogliati Australia Efficacy DSM-IV PD Not stated 41 79 RCT iCBT (disorder 5 8 Email, phone; PD: PDSS-SR
2016 (MINI) specific): 73 psychologists,
Transdiagnostic CBT-trained
iCBT: 72 counsellor,
research
assistant
Kiropoulos Australia Efficacy DSM-IV PD 59 39 72 RCT iCBT: 46 6 12 Email, phone; PD: PDSS-SR
2008 (ADIS-IV) Face to face psychologists, Ag: ACQ
CBT: 40 trainee
psychologists
Klein 2006 Australia Efficacy DSM-IV PD 82 Not 80 RCT iCBT: 19 6 6 Email; PD: PDSS
(ADIS-IV) stated Manualised psychologist, Ag: CSR
workbook:18 trainee
IC: 18 psychologists
(Continued)
Table 1. (Continued).
Mean % # of Length in Support type; Primary
Study Country Context Participants % with Ag age female Design Treatment arm: n modules weeks clinician type Measures
Richards Australia Efficacy DSM-IV PD Not stated Females: 56 Open trial iCBT: 9 5 5-8 Phone; not PD: PDSS
2002 (Prime MD & 30 specified
ADIS-IV) Males:
41
Richards Australia Efficacy DSM-IV PD 78 37 31 RCT iCBT: 12 6 8 Email; clinical PD: PDSS
2006 (ADIS-IV) iCBT (+ 6 psychologist, Ag: CSR
modules on trainee
stress): 11 psychologists
IC: 9
Ruwaard The Efficacy DSM-IV PD 57 38 72 RCT iCBT: 31 7 11 Comments on PD: PDSS-SR
2010 Netherlands symptoms (symptoms) WLC: 27 online Ag: MI-A
(interview) workbook;
Psychologists,
trainee
psychologists
Wims Australia Efficacy DSM-IV PD 100 44 60 Open trial iCBT: 10 6 9 Email, online PD: PDSS
2008 (MINI) discussion Ag: MI-A
forum;
psychiatry
registrar
Wims Australia Efficacy DSM-IV PD Not stated 42 76 RCT iCBT: 32 6 8 Email; psychiatry PD: PDSS
2010 (MINI) WLC: 27 registrar Ag: MI-A
Effectiveness studies
Allen 2016 Australia Effectiveness PD symptoms Not stated 39 64 Open trial iCBT: 330 5 12 Email; prescribing PD: PDSS-SR
Study 2* (PDSS-SR: clinician
81% ≥ 8) (various health
professional)
Bergstrom Sweden Effectiveness DSM-IV PD 100 34 55 Open trial iCBT: 20 10 10 Email; clinical PD: PDSS
2009 (MINI) psychologists Ag: MI-A
Bergstrom Sweden Effectiveness DSM-IV PD iCBT: 86 CBT: 34 iCBT: 64 RCT iCBT: 50 10 10 Email; PD: PDSS
2010 (MINI) Group: Group: Group: Face to face psychologist
83 35 59 group CBT: 54
Hadjistavro- Canada Effectiveness DSM-IV PD Not stated 40 69 Open trial iCBT: 25 12 12 Email, phone; PD: PDSS-SR
poulos (MINI) community
2014 therapists,
COGNITIVE BEHAVIOUR THERAPY

trainee
therapists
Hedman Sweden Effectiveness DSM-IV PD Not stated 37 61 Open trial iCBT: 570 10 10-12 Phone, messages; PD: PDSS-SR
2013 (MINI) psychologists,
11

trainee
psychologists
(Continued)
 12

Table 1. (Continued).
Mean % # of Length in Support type; Primary
Study Country Context Participants % with Ag age female Design Treatment arm: n modules weeks clinician type Measures
Klein Australia Effectiveness DSM-IV PD 33 37 83 Open trial iCBT: 6 9 6 None PD: CSR
2008 (ADIS-IV) Ag: CSR
Klein Australia Effectiveness DSM-IV PD 74 39 82 RCT iCBT with frequent 11 8 Email; PD: PDSS-SR
2009 (ADIS-IV) contact: 28 psychologists, Ag: CSR
iCBT with trainee
E. P. STECH ET AL.

infrequent psychologist
contact: 29
Mason Australia Effectiveness DSM-IV PD Not stated 35 67 Non- iCBT after 6 13 Email, phone; PD: PDSS-SR
2014 (Psychiatrist randomized psychiatrist clinical
diagnosis or trial assessment: 28 psychologist,
PDSS-SR >8) iCBT after psychiatry
automated registrar
assessment: 20
Nordgreen Norway Effectiveness DSM-IV PD Not stated 41 70 Open trial iCBT: 27 10 10 Phone; therapist PD: BSQ
2010* (SCID) Ag: MI-A
Nordgreen Sweden Effectiveness DSM-IV PD Not stated 36 65 Open trial iCBT: 114 9 Not stated Email; therapists PD: BSQ
2018* (MINI) Ag: MI-A
Ruwaard The Effectiveness PD symptoms Not stated 37 62 Open trial iCBT: 139 7 11 Email; clinical PD: PDSS-SR
2012* Netherlands (PDSS-SR; psychologists,
73% > 8) trainee clinical
psychologists
Shandley Australia Effectiveness DSM-IV PD 75 41 76 Non- iCBT with 6 core 12 Email; PD: PDSS
2008 (ADIS-IV) randomized Psychologist: 30 psychologists Ag: MI-A
trial iCBT with or
General Face to face
Practitioner: 45 appointments;
general
practitioners
Van The Effectiveness PD symptoms 42 37 68 RCT iCBT: 63 6 12 Email; trainee PD: PDSS-SR
Balle- Netherlands (PDSS-SR IC: 63 clinical
gooigen from 5-15) psychologists
2013
Note: Ag: Agoraphobia; PD: panic disorder; DSM-IV: Diagnostic and Statistical Manual of Mental Disorder, Fourth Edition. Measures: ADIS-IV: Anxiety Disorders Interview Schedule for DSM-IV;
ACQ: Agoraphobic Cognitions Questionnaire; BSQ: Body Sensations Questionnaire; CSR: ADIS-IV Clinician Severity Rating (for Panic Disorder or Agoraphobia, as relevant); MI-A: Mobility
Inventory—Alone subscale; MINI: Mini-International Neuropsychiatric Interview; PDSS-SR: Panic Disorder Severity Scale—Self-report; PDSS: Panic Disorder Severity Scale; PDSS-SR: Panic
Disorder Severity Scale—Self-Report; Prime MD: Primary Care Evaluation of Mental Disorders; SCID: Structured Clinical Interview for DSM-IV. Trial types: RCT: Randomised Control Trial.
Treatments: CBT: cognitive behavioural therapy; iCBT: internet-delivered cognitive behavioural therapy; IC: information control; WLC: waiting list control. * = study reported adverse effects.
All references for these studies can be found in the Supplementary Materials, Appendix C.
Table 2. Risk of bias in RCTs.
Risk of bias
Random sequence Allocation Blinding of participants Blinding of outcome Incomplete outcome Selective High Risk
Study generation concealment and personnel assessment data reporting Overall
Efficacy studies
Allen 2016 Study 1 - - + + - - N
Carlbring 2001 - - + + - ? N
Carlbring 2003 - ? - + - ? N
Carlbring 2005 - ? + - - ? N
Carlbring 2006 - - + - - ? N
Ciuca 2018 - - + - - - N
Fogliati 2016 - - + + - - N
Kiropoulos 2008 - - + + - ? N
Klein 2006 + - + ? - ? Y
Richards 2006 ? ? + ? - ? Y
Ruwaard 2010 ? - + + - ? Y
Wims 2010 - - + + - - N
Effectiveness studies
Bergstrom 2010 - - + - - ? N
Klein 2009 - - + - - ? N
Van Ballegooigen - - + + - ? N
2013
Note: - = low risk, + = high risk of bias, and ? = unclear risk of bias. A study was classified as high risk overall if the risk was rated as high or unclear in more than three domains. All references
for these studies can be found in the Supplementary Materials, Appendix C.
COGNITIVE BEHAVIOUR THERAPY
13
14 E. P. STECH ET AL.

Risk of bias within non-randomized and open trials

None of the non-randomized or open trials reported a trial registration number or
published protocol. All studies were coded as high risk for measurement of outcomes,
due to reliance on self-report measures and an inability for assessors in open trials to be
blind to intervention. The automated nature of iCBT ensured minimal risk of deviation
from intended interventions, and missing data was appropriately handled in most
studies (n = 10; 83%). Two studies (17%) were classified as high risk overall, since
more than three domains were rated as having high or unclear risk (see Table 3).

Primary outcomes
Between-group effects of iCBT versus control groups at post-treatment
We compared the effects of iCBT with waitlist or information controls (see Figures 2 and 3
for forest plots of panic and agoraphobia, respectively). The overall between-group effect
size for panic symptom severity was large (n = 9, g = 1.22, 95%CI: .72–1.71), with high and
significant heterogeneity (Q = 52.65; I 2 = 85). The overall effect for agoraphobia symptom
severity was large (n = 6, g = .91, 95%CI: .36–1.45), with high and significant heterogeneity
(Q = 22.50; I 2 = 78). Sensitivity analyses excluding studies with high risk of bias overall
found smaller effect sizes for both panic (n = 6, g = 1.04, 95%CI: .52–1.55; Q = 28.53, I2 = 82)
and agoraphobia symptom symptoms (n = 3, g = .64, 95%CI: .16–1.13; Q = 4.53, I2 = 56).

Between-group effects of iCBT versus face-to-face CBT at post-treatment

Analyses revealed no significant difference between iCBT and face-to-face CBT for
panic symptoms (n = 3, g = .14, 95%CI: −.12-.40), but there was a significant difference
in favor of iCBT for agoraphobia (n = 2, g = .38, 95%CI: .03-.73, p =.03), with minimal
heterogeneity (Q = .007; I 2 = 0) among these effects. Due to the small number of studies
comparing iCBT and face-to-face CBT, these results should be interpreted with caution.

Remission and clinical improvement

There was insufficient information reported in most studies to analyze the proportion
of participants in remission or clinically improved following iCBT.

Secondary outcomes
Effects of iCBT programs on primary outcomes between pre- and post-treatment
(within-group effect sizes)
To calculate the average within-group effect sizes between pre- and post-treatment, we
included 29 comparisons (N = 1587), as three studies included multiple iCBT conditions
that were eligible. Analysis of studies that reported pre-to post-treatment effects found
a mean within-group effect size (Hedges g) of 1.16 for panic symptoms (n = 29, 95%CI:
.98–1.33), with high and significant heterogeneity (Q = 172.37; I 2 = 84), and 0.73 for
agoraphobia symptoms (n = 18, 95%CI: .56-.90), with moderate and significant hetero-
geneity (Q = 43.68; I 2 = 61). Sensitivity analyses that excluded studies with high risk of bias
found similar effect sizes for panic (n = 23, g = 1.11, 95%CI: .92–1.30; Q = 143.25, I2 = 84)
Table 3. Risk of bias in non-randomized and open trials.
Risk of bias
Selection of Intervention Missing Measurement of Selective
Study Confounding participants Misclassification deviation data outcomes reporting High Risk Overall
Efficacy studies
Richards 2002 - - - - + + ? N
Wims 2008 - - - - - + ? N
Effectiveness studies
Allen 2016 Study 2 - - - - - + ? N
Bergstrom 2009 - - - - - + ? N
Hadjistavropoulos - - - - - + ? N
2014
Hedman 2013 - - - - - + ? N
Klein 2008 - - - - - + ? N
Mason 2014 - + - - + + ? Y
Nordgreen 2010 - - - - - + ? N
Nordgreen 2018 - - - - - + ? N
Ruwaard 2012 - - - - - + ? N
Shandley 2008 + + - - - + ? Y
Note: - = low risk or not applicable, + = high risk of bias, and ? = unclear risk of bias due to lack of information. A study was classified as high risk overall if the risk was rated as high or
unclear in more than three domains. All references for these studies can be found in the Supplementary Materials, Appendix C.
COGNITIVE BEHAVIOUR THERAPY
15
16 E. P. STECH ET AL.

Figure 2. Forest plot of between-group effect of iCBT vs waitlist or information controls on panic
symptom severity at post-treatment. Note: combined = two iCBT interventions were combined to
create one pairwise comparison to a control.

Figure 3. Forest plot of between-group effect of iCBT vs waitlist or information controls on

agoraphobia symptom severity at post-treatment. Note: combined = two iCBT interventions were
combined to create one pairwise comparison to a control.

and agoraphobia (n = 14, g = 0.75, 95%CI: .57-.93; Q = 31.85, I2 = 59) symptoms. Forest
plots of the impact of iCBT on panic and agoraphobia at post-treatment are presented in
Appendix D and E, respectively.

Effects of iCBT programs on panic and agoraphobia between pre-treatment and

follow-up (within-group effect sizes)
Twelve and seven studies reported data at 3–6 month follow-up for panic and agor-
aphobia symptoms, respectively. Pre-treatment to 3–6 month follow-up within-group
effect sizes were, on average, large for panic (n = 14, g = 1.59, 95%CI: 1.19–1.99) and
agoraphobia (n = 7, g = 1.26, 95%CI: .84–1.67). Heterogeneity was high and significant
across both measures (panic: Q = 131.48, I 2 = 90; agoraphobia: Q = 28.78, I 2 = 79).
Sensitivity analyses excluding studies with high risk of bias found a similar results for
panic (n = 10, g = 1.49, 95%CI: 1.03–1.96; Q = 107.56, I2 = 92), and agoraphobia (n = 3,
g = 1.17, 95%CI: .87–1.48) symptoms; although the heterogeneity for agoraphobia
measures was reduced (Q = 3.08, I2 = 35). An insufficient number of studies (n = 3)
reported outcome data at 1–2 years follow-up for the data to be analyzed.
 COGNITIVE BEHAVIOUR THERAPY 17

Subgroup analyses of within-group pre- to post-treatment effects

See Table 4 for results of subgroup analyses.

Efficacy versus effectiveness studies. The average within-group effect sizes were larger
for panic symptoms in efficacy studies compared to effectiveness studies (Qbet = 4.53,
df = 1, p < .001), but there was no significant differences for agoraphobia severity.
(Qbet = .004, df = 1, p =.95).

Inclusion of arousal reduction techniques. There were no statistically significant differ-

ences in the average within-group effect size obtained for panic between programs that
included arousal reduction techniques and those that did not (Qbet = .08, df = 1, p =.77).
There was an insufficient number of studies (n = 2) to compare effect sizes for agoraphobia.

Number of treatment modules. We coded each study into less than or equal to six
modules, and more than six modules. There was no significant differences in the effect
sizes for panic (Qbet = .33, df = 1, p =.57) or agoraphobia (Qbet = .01, df = 1, p =.92).

Degree of clinician support. We coded each study into less than one hour, one to two
hours, or more than two hours of clinician support during the intervention. Self-guided
interventions were excluded. There were no significant differences in the effect sizes for
panic (Qbet = 2.98, df = 2, p =.23) based on degree of clinician support, and insufficient
data to compare the effect sizes for agoraphobia.

Adherence
Out of 27 studies, 19 (70%) reported data on adherence. Most studies (n = 13) reported
the mean number of completed modules, which is difficult to compare across studies
due to variability in the total number of modules included in iCBT programs. Many
studies (n = 11) also reported the percentage of participants who completed the entire
iCBT program, however these values were highly variably between studies, ranging
from 8% to 84%.
Unfortunately, there was insufficient adherence data to estimate the overall mean
proportion of participants who were introduced to exposure during the intervention. Of
the studies that reported when exposure was first introduced (n = 13), most iCBT
programs (n = 10) began exposure in module 4, which in most cases was in the second
half of the program.

Table 4. Sub-group analyses for pre-post treatment effects.

Panic symptoms Agoraphobia symptoms
Variable Subgroups n g 95% CI p n g 95% CI p
Context Efficacy 15 1.38 1.09–1.67 < .001 11 .73 .48-.99 .95
Effectiveness 14 .98 .76–1.20 7 .74 .52-.97
Arousal reduction strategies Included 24 1.18 .98–1.38 .77 Insufficient data
Not included 5 1.11 .66–1.55
Number of treatment modules ≤ 6 modules 14 1.10 .87–1.33 .57 9 .73 .43–1.04 .92
> 6 modules 15 1.20 .94–1.47 9 .75 .57-.93
Degree of clinician support <1 hour 6 1.27 .85–1.68 .23 Insufficient data
1–2 hours 5 .93 .66–1.21
>2 hours 12 1.25 .94–1.57
18 E. P. STECH ET AL.

Adverse effects
Only five studies described data on symptom worsening or other adverse effects. Three
studies reported no participants experienced symptoms worsening at post-treatment;
one reported 2.6% of participants who received iCBT experienced symptom worsening;
and the last stated that 5% of participants reported a negative impact of treatment.

Publication bias
Based on inspection of the funnel plot and using Duval and Tweedie’s Trim and Fill
procedure, there was no evidence of publication bias for panic or agoraphobia out-
comes for RCTs comparing iCBT to controls (no studies were removed). However,
there is still a risk that publication bias exists and was not detected by this method.

Discussion
In the current study, we systematically reviewed the literature on internet-delivered
cognitive behavioral therapy (iCBT) for panic symptoms with or without agoraphobia
in adults. We identified 27 studies (with 2590 participants in total), including 15 RCTs.
We found large within-group improvements in panic severity between pre- to post-
treatment (g = 1.16). Results from the nine RCTs comparing iCBT with waitlist or
information control conditions showed large mean between-group differences for panic
severity (g = 1.22), and results from three RCTs suggested that iCBT had similar results
to face-to-face CBT in reducing panic severity. These findings are comparable with the
effect sizes found in previous reviews of iCBT (Andrews et al., 2018; Carlbring et al.,
2018; Olthuis et al., 2015).
Past research shows a clear association between agoraphobia and severity of impairment
(Wittchen et al., 2010) and poor long-term outcomes (Bruce et al., 2005). This is the first
review to assess the impact of iCBT on agoraphobia symptom severity. Fifteen (62%) of the
27 studies included a measure of agoraphobic avoidance. We found medium reductions in
agoraphobia severity from baseline to post-treatment (g = 0.73), and large between-group
differences for iCBT relative to WLC and information control groups in reducing agor-
aphobia (g = .91). Only two RCTs compared the effects of iCBT and face-to-face CBT on
agoraphobia. Together these findings show that iCBT for panic disorder improves agor-
aphobia, albeit to a lesser degree than the reductions observed in panic severity. Further
research is needed to explore optimal ways of targeting agoraphobic avoidance in iCBT.
It is important to consider these findings in light of the quality of the included studies
and the inherent risk of bias in some. The quality of many RCTs was unclear after we
assessed bias risk, primarily due to the lack of trial pre-registration and/or published
protocols. Additionally, it is difficult to achieve blinding of participants and personnel in
psychotherapy intervention trials, and many trials did not include a blinded outcome
measure, meaning demand characteristics may have influenced the results. Sensitivity
analyses suggest that the inclusion of low quality RCTs may have led to an overestimate
of treatment effects. After excluding low quality RCTs we found slightly lower between-
group effect sizes—still favouring iCBT over WLC and information controls—for panic
(g = .91) and agoraphobia (g = .64). These findings suggest that the quality of RCTs
influenced the size of the observed effects, especially for agoraphobia.
 COGNITIVE BEHAVIOUR THERAPY 19

This is the first comprehensive review of both efficacy and effectiveness studies of
iCBT for panic disorder. Of the 27 studies included in this review, 14 were effec-
tiveness studies conducted in routine or usual care settings. The results suggest
positive outcomes for iCBT on panic and agoraphobia symptoms not only in tightly
controlled research settings, but also usual care settings. Although we found slightly
lower reductions in panic severity in effectiveness studies versus efficacy studies, the
effect sizes for effectiveness trials were still large, and there were no differences in
agoraphobia outcomes. Unfortunately, there is a paucity of effectiveness RCTs, which
is a major limitation of this research literature. Two exceptions are Bergstrom et al.
(2010), who reported equivalent outcomes for iCBT and group CBT; and Van
Ballegooijen et al. (2013), who reported no difference between iCBT and an infor-
mation control, due to poor adherence to iCBT. Adherence may be a key factor that
differentiates efficacy and effectiveness trials, which may be due to limited resources
(including clinician time) in community based services. There was limited adherence
data available in many studies; however, the available data suggests there was
a substantial degree of variation between studies in the proportion of participants
who completed all modules of the iCBT programs (ranging from 8% to 84%). It may
be fruitful for future research to explore what program factors strengthen adherence
in usual care settings.
Although we found large effects of iCBT on panic and agoraphobia symptoms, there
was significant variation in the effect sizes found across studies, in line with previous
reviews (Olthuis et al., 2015). We conducted subgroup analyses to explore which factors
may account for this high heterogeneity. Readers should interpret the subgroup analyses
with caution due to confounding factors potentially associated with the categorization of
subgroups. For example, length of a treatment may correlate with other variables such as
the amount of clinician support received or adherence. Results of these analyses failed to
find any effect of treatment length (more or less than 6 modules), or the inclusion/
exclusion of arousal reduction techniques. It is possible that the introduction of core CBT
elements common to all programs (psychoeducation, cognitive restructuring and expo-
sure) was sufficient, regardless of “dose” or additional arousal reduction techniques.
Indeed, a recent component meta-analysis found limited benefit in including breathing
retraining or progressive muscle relaxation in CBT for panic disorder (Pompoli et al.,
2018). Alternatively, the impact of treatment components and length may have been
masked by the considerable variation in adherence; however, this could not be assessed
due to limited and inconsistent reporting of adherence data across studies. Considering
the high variability in adherence, iCBT developers should consider what treatment
components to prioritize, and emphasize those in the earliest lessons. Dismantling studies
would help inform optimal ordering of treatment components to maximize the efficiency
and effectiveness of iCBT programs.
We also did not find a significant difference in effect sizes depending on the degree of
clinician support provided (comparing those who received less than 1 hour, 1–2 hours or
more than 2 hours of support). This is inconsistent with our hypothesis and past
assertions within the field that higher degree of support is associated with larger outcomes
(Palmqvist, Carlbring, & Andersson, 2007). It should be noted that earlier research tended
to compare iCBT with no support to iCBT with a substantial amount of clinician support
(1.5–5 hours). Programs have become more sophisticated over time, with automated
20 E. P. STECH ET AL.

contact that reduces need for individualized clinician contact. More recent research has
suggested the impact of therapist guidance may not be as great as previously thought,
especially for newer generation iCBT programs (Baumeister, Reichler, Munzinger, & Lin,
2014; Fogliati et al., 2016).
The nature of the sample in each study is an alternate factor that may have
contributed to the unexplained heterogeneity in effect sizes across studies. Most studies
utilized small sample sizes that may not be representative. Not all studies required
a formal diagnosis of panic disorder, and those that did varied in their diagnostic
procedures. Samples that were subclinical or less severe at baseline had less room for
improvement (e.g., Ruwaard, Broeksteeg, Schrieken, Emmelkamp, & Lange, 2010; Van
Ballegooijen et al., 2013). Additionally, studies did not consider the impact of comorbid
diagnoses on treatment effect, despite high rates of comorbidity among studies that
reported detailed diagnostic data at baseline. An important recent trend in the field is
the growth of transdiagnostic iCBT programs aimed at accommodating such comor-
bidity (Păsărelu, Andersson, Nordgren, & Dobrean, 2016). Initial studies have not
found significant differences between transdiagnostic and disorder-specific iCBT in
the context of panic (Fogliati et al., 2016), however further research is needed to identify
what diagnostic patterns or other factors moderate effects.

Limitations
The findings should be interpreted with acknowledgement of several limitations. First,
despite systemically searching several databases, we may have unintentionally omitted
eligible articles. We attempted to address this by examining the reference lists of
relevant articles and previous meta-analyses, however we did not search for unpub-
lished studies or grey literature. We also only included studies published in
English. Second, to increase the sample size for each analyses, we combined self-
report measures and clinician-rated measures, which may be a source of systematic
variation between studies. Thirdly, the parameters for classifying studies by degree of
clinician support and treatment length were defined post-hoc to ensure a sufficient
number of studies in each grouping. Fourth, within-group effect sizes should also be
interpreted with caution since these analyses do not control for confounding variables
such as spontaneous remission, placebo effects or regression to the mean. Finally, this
review focused solely on iCBT programs, and excluded smartphone apps.

Future directions
This systematic review also highlighted several limitations of the field that are impor-
tant avenues for future research. The majority of studies were performed in two
countries, by a handful of research groups. More independent replication studies across
a broader range of cultural contexts are needed. Only five studies reported data on
adverse effects of iCBT. There was limited long-term data on the effects of iCBT for
panic and especially agoraphobia severity, which is a limitation of the iCBT literature
across all disorders (Andersson, Rozental, Shafran, & Carlbring, 2018). Additionally,
there were limited trials comparing iCBT to face-to-face CBT, transdiagnostic iCBT, or
alternative internet delivered therapy, and no trials comparing iCBT to
 COGNITIVE BEHAVIOUR THERAPY 21

pharmacotherapy. Future studies should seek to address these gaps in the literature.
Future reviews are also needed to examine the literature on smartphone apps for panic
disorder.

Conclusion
In conclusion, this meta-analysis provides evidence that iCBT for panic is not only
efficacious, but also effective in usual care settings. This is also the first meta-analysis to
provide evidence that iCBT has large effects on agoraphobia symptoms and outper-
forms control groups in reducing agoraphobia severity. These findings suggest that
clinician-guided iCBT is appropriate for use in routine care settings, and may be an
effective avenue for increasing the availability of evidence-based interventions for panic
disorder and agoraphobia within the community. Further high quality RCTs are now
needed to refine treatment protocols, compare iCBT to other treatment options, and
provide best-practice guidelines for implementing iCBT in usual care settings.

Notes
1. https://www.crd.york.ac.uk/prospero/.
2. We conducted a sensitivity analysis by recalculating the estimated effects using correlation
values of 0.25 and 0.75. There were no substantial differences in the effect size estimates
when we used these alternate within-group correlations, with the largest difference
between estimates being 0.05.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This work was supported by an Australian Government Research Training Program Scholarship
granted to Eileen Stech; and an Australian National Medical Research Council and Medical
Research Future Fund Career Development Fellowship awarded to Jill Newby (MRFF1145382).

ORCID
Eileen P. Stech http://orcid.org/0000-0003-0151-9026
Jaclyn Lim http://orcid.org/0000-0001-5176-8413
Emily L. Upton http://orcid.org/0000-0002-8351-2591
Jill M. Newby http://orcid.org/0000-0002-6473-9811

References
American Psychiatric Association. (2009). Practice guideline for the treatment of patients with
panic disorder, 2nd ed. Retrieved from http://psychiatryonline.org/pb/assets/raw/sitewide/prac
tice_guidelines/guidelines/panicdisorder.pdf
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders,
5th edition (DSM-5). Washington, DC: Author.
22 E. P. STECH ET AL.

Andersson, G., Rozental, A., Shafran, R., & Carlbring, P. (2018). Long-term effects of
internet-supported cognitive behavior therapy. Expert Review of Neurotherapeutics, 18(1),
1–8. doi:10.1080/14737175.2018.1400381
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M.
(2018). Computer therapy for the anxiety and depression disorders is effective, acceptable and
practical health care: An updated meta-analysis. Journal of Anxiety Disorders, 5(10), e13196.
doi:10.1016/j.janxdis.2018.01.001
Andrews, G., Newby, J. M., & Williams, A. D. (2015). Internet-delivered cognitive behavior
therapy for anxiety disorders is here to stay. Current Psychiatry Reports, 17(1), 533.
doi:10.1007/s11920-014-0533-1
Balk, E. M., Earley, A., Patel, K., Trikalinos, T. A., & Dahabreh, I. J. (2012). AHRQ methods for
effective health care empirical assessment of within-arm correlation imputation in trials of
continuous outcomes. Rockville (MD): Agency for Healthcare Research and Quality (US).
Barrera, T. L., & Norton, P. J. (2009). Quality of life impairment in generalized anxiety disorder,
social phobia, and panic disorder. Journal of Anxiety Disorders, 23(8), 1086–1090. doi:10.1016/
j.janxdis.2009.07.011
Baumeister, H., Reichler, L., Munzinger, M., & Lin, J. (2014). The impact of guidance on
Internet-based mental health interventions - A systematic review. Internet Interventions, 1
(4), 205–215. doi:10.1016/j.invent.2014.08.003
Bergstrom, J., Andersson, G., Ljotsson, B., Ruck, C., Andreewitch, S., Karlsson, A., . . .
Lindefors, N. (2010). Internet-versus group-administered cognitive behaviour therapy for
panic disorder in a psychiatric setting: A randomised trial. BMC Psychiatry, 10, 54.
doi:10.1186/1471-244X-10-54
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to
fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2),
97–111. doi:10.1002/jrsm.12
Bruce, S. E., Yonkers, K. A., Otto, M. W., Eisen, J. L., Weisberg, R. B., Pagano, M., . . .
Keller, M. B. (2005). Influence of psychiatric comorbidity on recovery and recurrence in
generalized anxiety disorder, social phobia, and panic disorder: A 12-year prospective study.
American Journal of Psychiatry, 162(6), 1179–1187. doi:10.1176/appi.ajp.162.6.1179
Carlbring, P., Andersson, G., Cuijpers, P., Riper, H., & Hedman-Lagerlöf, E. (2018). Internet-
based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: An
updated systematic review and meta-analysis. Cognitive Behaviour Therapy, 47(1), 1–18.
doi:10.1080/16506073.2017.1401115
Ciuca, A. M., Berger, T., Crişan, L. G., & Miclea, M. (2018). Internet-based treatment for panic
disorder: A three-arm randomized controlled trial comparing guided (via real-time video
sessions) with unguided self-help treatment and a waitlist control. PAXPD study results.
Journal of Anxiety Disorders, 56(July2017), 43–55. doi:10.1016/j.janxdis.2018.03.009
Cohen, J. (1988). Statistical power analysis for behavioral sciences (2nd ed.). Hillsdale, NJ:
Erlbaum.
Duval, S., & Tweedie, R. (2000a). A nonparametric “trim and fill” method of accounting for
publication bias in meta-analysis. Journal of the American Statistical Association, 95(449),
89–98. doi:10.1080/01621459.2000.10473905
Duval, S., & Tweedie, R. (2000b). Trim and fill: A simple funnel-plot-basedmethod of test- ing
and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. doi:10.2307/
2676988
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected
by a simple, graphical test. Bmj, 315(7109), 629–634. doi:10.1136/bmj.315.7109.629
Fogliati, V. J., Dear, B. F., Staples, L. G., Terides, M. D., Sheehan, J., Johnston, L., . . . Titov, N.
(2016). Disorder-specific versus transdiagnostic and clinician-guided versus self-guided
internet-delivered treatment for panic disorder and comorbid disorders: A randomized con-
trolled trial. Journal of Anxiety Disorders, 39, 88–102. Journal Article. doi:10.1016/j.
janxdis.2016.03.005.
 COGNITIVE BEHAVIOUR THERAPY 23

Goisman, R. M., Warshaw, M. G., & Keller, M. B. (1999). Psychosocial treatment prescriptions
for generalized anxiety disorder, panic disorder, and social phobia, 1991–1996. The American
Journal of Psychiatry, 11(November), 1819–1821.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic
Press.
Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interven- tions.
Retrieved from www.cochrane-handbook.org
Klein, B., Shandley, K., Austin, D., & Nordin, S. (2008). A pilot trial of “panic online” as a self-
guided treatment for panic disorder. E-Journal of Applied Psychology, 4(2), Journal Article,
25–30. doi:10.7790/ejap.v4i2.136
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G.; The PRISMA Group. (2009). Preferred
reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS
Medicine, 6(7), e1000097.
National Institute for Clinical Excellence. (2011). Clinical guidelines 123. Retrieved from https://
www.nice.org.uk/guidance/cg123/
Nordgreen, T., Gjestad, R., Andersson, G., Carlbring, P., & Havik, O. E. (2018). The implemen-
tation of guided internet-based cognitive behaviour therapy for panic disorder in a
routine-care setting: Effectiveness and implementation efforts. Cognitive Behaviour Therapy,
47(1), 62–75. doi:10.1080/16506073.2017.1348389
Norton, P. J., & Price, E. C. (2007). A meta-analytic review of adult cognitive-behavioral
treatment outcome across the anxiety disorders. The Journal of Nervous and Mental Disease,
195(6), 521–531. doi:10.1097/01.nmd.0000253843.70149.9a
Olthuis, J. V., Watt, M. C., Bailey, K., Hayden, J. A., & Stewart, S. H. (2015). Therapist-supported
internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database of
Systematic Reviews, 3(3), CD011565. doi:10.1002/14651858.CD011565
Palmqvist, B., Carlbring, P., & Andersson, G. (2007). Internet-delivered treatments with or
without therapist input: Does the therapist factor have implications for efficacy and cost?
Expert Review of Pharmacoeconomics & Outcomes Research, 7(3), 291–297. doi:10.1586/
14737167.7.3.291
Păsărelu, C. R., Andersson, G., Nordgren, L. B., & Dobrean, A. (2016). Internet-delivered
transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression:
A systematic review and meta-analysis of randomized controlled trials. Cognitive Behaviour
Therapy, 46(1), 1–28. doi:10.1080/16506073.2016.1231219
Pompoli, A., Furukawa, T. A., Efthimiou, O., Imai, H., Tajika, A., & Salanti, G. (2018).
Dismantling cognitive-behaviour therapy for panic disorder: A systematic review and compo-
nent network meta-analysis. Psychological Medicine, 48(12), 1945–1953. doi:10.1017/
S0033291717003919
Pompoli, A., Furukawa, T. A., Imai, H., Tajika, A., Efthimiou, O., & Salanti, G. (2016).
Psychological therapies for panic disorder with or without agoraphobia in adults: A network
meta-analysis. The Cochrane Database of Systematic Reviews, 4(4), CD011004. doi:10.1002/
14651858.CD011004.pub2
Richards, J. C., & Alvarenga, M. E. (2002). Extension and replication of an internet-based
treatment program for panic disorder. Cognitive Behaviour Therapy, 31(1), Journal Article,
41–47. doi:10.1080/16506070252823652
Ruwaard, J., Broeksteeg, J., Schrieken, B., Emmelkamp, P., & Lange, A. (2010). Web-based
therapist-assisted cognitive behavioral treatment of panic symptoms: A randomized controlled
trial with a three-year follow-up. Journal of Anxiety Disorders, 24(4), 387–396. doi:10.1016/j.
janxdis.2010.01.010
Schmidt, N. B., Woolaway-Bickel, K., Trakowski, J., Santiago, H., Storey, J., Koselka, M., &
Cook, J. (2000). Dismantling cognitive-behavioral treatment for panic disorder: Questioning
the utility of breathing retraining. Journal of Consulting and Clinical Psychology, 68(3),
417–424. doi:10.1037/0022-006X.68.3.417
24 E. P. STECH ET AL.

Sherbourne, C. D., Wells, K. B., & Judd, L. L. (1996). Functioning and well-being of patients
with panic disorder. American Journal of Psychiatry, 153(2), 213–218. doi:10.1176/
ajp.153.2.213
Stein, M. B., Roy-Byrne, P. P., Craske, M. G., Campbell-Sills, L., Lang, A. J., Golinelli, D., . . .
Sherbourne, C. D. (2011). Quality of and patient satisfaction with primary health care for
anxiety disorders. The Journal of Clinical Psychiatry, 72(7), 970–976. doi:10.4088/
JCP.09m05626blu
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., . . .
Higgins, J. P. T. (2016). ROBINS-I: A tool for assessing risk of bias in non-randomised studies
of interventions. Bmj, 355, i4919. doi:10.1136/bmj.i4919
Van Ballegooijen, W., Riper, H., Klein, B., Ebert, D. D., Kramer, J., Meulenbeek, P., &
Cuijpers, P. (2013). An Internet-based guided self-help intervention for panic symptoms:
Randomized controlled trial. Journal of Medical Internet Research, 15(7), 1–12. doi:10.2196/
jmir.2362
Wittchen, H. U., Gloster, A. T., Beesdo-Baum, K., Fava, G. A., & Craske, M. G. (2010).
Agoraphobia: A review of the diagnostic classificatory position and criteria. Depression and
Anxiety, 27(2), 113–133. doi:10.1002/da.20646
Wolitzky-Taylor, K. B., Fenwick, K., Lengnick-Hall, R., Grossman, J., Bearman, S. K., Arch, J., . . .
Chung, B. (2018). A preliminary exploration of the barriers to delivering (and receiving)
exposure-based cognitive behavioral therapy for anxiety disorders in adult community mental
health settings. Community Mental Health Journal, doi:10.1007/s10597-018-0252-x
Wolitzky-Taylor, K. B., Zimmermann, M., Arch, J. J., De Guzman, E., & Lagomasino, I. (2015).
Has evidence-based psychosocial treatment for anxiety disorders permeated usual care in
community mental health settings? Behaviour Research and Therapy, 72, 9–17. doi:10.1016/j.
brat.2015.06.010