Cognitive Behaviour Therapy

ISSN: 1650-6073 (Print) 1651-2316 (Online) Journal homepage: https://www.tandfonline.com/loi/sbeh20

Web-based CBT for the prevention of anxiety

symptoms among medical and health science
graduate students

Ashley N. Howell, Alyssa A. Rheingold, Thomas W. Uhde & Constance Guille

To cite this article: Ashley N. Howell, Alyssa A. Rheingold, Thomas W. Uhde & Constance
Guille (2019) Web-based CBT for the prevention of anxiety symptoms among medical
and health science graduate students, Cognitive Behaviour Therapy, 48:5, 385-405, DOI:
10.1080/16506073.2018.1533575

To link to this article: https://doi.org/10.1080/16506073.2018.1533575

Published online: 11 Dec 2018.

Submit your article to this journal

Article views: 418

View related articles

View Crossmark data

Citing articles: 2 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=sbeh20
COGNITIVE BEHAVIOUR THERAPY
2019, VOL. 48, NO. 5, 385–405
https://doi.org/10.1080/16506073.2018.1533575

Web-based CBT for the prevention of anxiety symptoms

among medical and health science graduate students
Ashley N. Howell , Alyssa A. Rheingold , Thomas W. Uhde
and Constance Guille
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina (MUSC),
Charleston, SC, USA

ABSTRACT ARTICLE HISTORY

Medical and health science graduate students report greater Received 3 November 2017
anxiety problems than the general population, but they are less Accepted 27 September 2018
likely to seek treatment due to cultural and logistical barriers. KEYWORDS
One preventative approach that overcomes these barriers is web- CBT; internet; web; anxiety;
based cognitive behavioral therapy (webCBT). It is unknown prevention; medical
whether webCBT is effective for preventing anxiety escalation students; graduate students
within this population. A randomized controlled trial was con-
ducted, comparing the effects of webCBT versus a control group
(CG). Medical university students (n=594; Mage=27; 67% female;
80% Caucasian) completed online baseline measures and four
assigned online activities. Measures were re-administered after
approximately three months. There was a small interaction effect
between time of assessment and treatment condition. Anxiety
severity was lower in the webCBT (M[SD]=2.88[3.36]) versus CG
condition (M[SD]=3.69 [3.35]) at follow-up. This effect was mod-
erate for students with mild, versus minimal, anxiety at baseline.
The proportion of students with possible anxiety disorder was
lower in the webCBT (4.5%) versus CG (8.5%) condition, and the
proportion of mildly anxious students with a clinically significant
increase in symptoms was lower in the webCBT (10%) versus CG
(20%) condition. WebCBT may aid in preventing anxiety escala-
tion in this population, particularly for at-risk students who report
mild anxiety symptoms.

Graduate students in health-related fields tend to experience increased stress and

anxiety compared with the general population and non-students of similar ages
(Dyrbye, Thomas, & Shanafelt, 2006, for a review). The (2013–2014) point preva-
lence rate of a positive screen for a possible anxiety disorder, per the 7-Item
Generalized Anxiety Disorder Scale (GAD-7 ≥ 10), was over eightfold higher in
medical students, residents, and fellows (19% vs. 2%) than nationally representative
age-matched controls (Mousa, Dhamoon, Lander, & Dhamoon, 2016). This compar-
ison data is generally consistent with a separate nationally representative sample not
exclusive to age, which found that a positive screen occurred in about 4% of
respondents.

CONTACT Constance Guille guille@musc.edu Department of Psychiatry and Behavioral Sciences, Medical
University of South Carolina, Charleston, SC 29425
© 2018 Swedish Association for Behaviour Therapy
386 A. N. HOWELL ET AL.

At the trait level, individuals who pursue post-baccalaureate education are likely to
report higher perfectionism and neuroticism than individuals who do not (Enns, Cox,
Sareen, & Freeman, 2001; Tyssen et al., 2007)—increasing risk and prevalence for
anxiety-related problems. In addition, graduate students are a vulnerable population
for anxiety escalation, because they are exposed to and report chronic stressors, such as
long work hours, limited income, burnout, frequent evaluation, and sleep deprivation
(Dyrbye et al., 2006; Mazzola, Walker, Shockley, & Spector, 2011; Myers et al., 2012;
Oswalt & Riddock, 2007). Unfortunately, a “critical deficiency in resilience education
and training” exists for medical school, residency, and graduate students (Beresin et al.,
2016, p. 9), and experts have called for programs to increase psychological resiliency
programs for this at-risk population (Rakesh, Pier, & Costales, 2017).
Graduate institutions, however, cite credible barriers to meeting these calls for
action, including limited staff and funding (Novotney, 2014) and, therefore, long wait-
ing lists if services are offered (Stecker, 2004). At the same time, the likelihood of these
students seeking professional help for anxiety or other concerns is lower than that of
the general population—creating a discrepancy between increased anxiety symptoms
and underutilized treatment options that are typically both on and off campus (Chew-
Graham, Rogers, & Yassin, 2003; Dyrbye et al., 2006). To illustrate, among people with
an anxiety disorder in the USA (Wang et al., 2005) or Canada (Roberge, Fournier,
Duhoux, Nguyen, & Smolders, 2011), 36.9% report receiving treatment within the past
year. In one sample predominantly comprised of doctoral students, about 31% reported
seeking mental health services for any problem at some point throughout their several
years of graduate school enrollment (Hyun, Quinn, Madon, & Lustig, 2006). While
service underutilization in the general population can be explained in some part by lack
of knowledge about mental illness or treatment options for anxiety (Thompson, Hunt,
& Issakidis, 2004; see also Gulliver, Griffiths, & Christensen, 2010; Sareen et al., 2007;
for samples nonspecific to anxiety), many medical and health science graduate students
learn about, promote, and even provide interventions for anxiety-related sequelae.
Major reasons for explicit service underutilization for medical and health science
graduate students include stigma about emotional problems among health providers
and/or academics; concerns about anonymity and confidentiality within local resources
at which colleagues may work; fear of negative impacts on one’s career and/or academic
record; time constraints; financial burden; and perceived inadequacy of mental health
sessions available at the university (Brimstone, Thistlethwaite, & Quirk, 2007; Dunn,
Iglewicz, & Moutier, 2008; Guille et al., 2015; Tjia, Givens, & Shea, 2005).
Overall, medical and health science graduate students are not only uniquely vulner-
able to anxiety due to particular traits and increased chronic stress, but they also
experience unique barriers to seeking professional help when anxiety and stress rise.
Thus, while research is certainly needed to improve mental health treatment-seeking
within this population, focusing resources on preventing maladaptive escalation of
anxiety symptoms may help to obviate higher risk of onset, and unnecessarily pro-
longed distress and interference, for students. According to a review by Reavley and
Jorm in 2010, there was limited to weak evidence that interventions (predominantly
face-to-face CBT) were effective in preventing anxiety disorders in higher education
students, at least in the longer term. More recently, Conley, Shapiro, Kirsch, and Durlak
(2017) conducted a meta-analysis of mental health prevention programs for at-risk
 COGNITIVE BEHAVIOUR THERAPY 387

higher education students; studies that targeted anxiety were conducted primarily
among undergraduates and for face-to-face interventions. They found that for students
with subclinical signs of mental health symptoms, targeted anxiety prevention programs
(e.g. CBT or relaxation skills training) resulted in a medium-to-large post-intervention
effect size, g = .67, CI(95) = .50-.84. Barriers to implementing these prevention
programs still exist, however, and the efficacy of anxiety prevention programs in
medical and health science graduate students, who face additional unique stressors
and barriers to prompt mental healthcare, is unclear.
One possible solution to preventing anxiety problems in this population is web-based
cognitive behavioral therapy (webCBT). WebCBT (for the current study: MoodGYM) is
anonymous, available at all hours to accommodate various schedules, and several
webCBT programs are free or of low cost. WebCBT has been shown to reduce anxiety
and other psychiatric problems in samples not exclusive to graduate students (for meta-
analytic reviews see: Adelman, Panza, Bartley, Bontempo, & Bloch, 2014; Cuijpers et al.,
2009; Griffiths, Farrer, & Christensen, 2010; review), with some evidence that webCBT
is comparable in corrective treatment outcomes to face-to-face therapy (Andersson,
Cuijpers, Carlbring, Riper, & Hedman, 2014; for a review; Adelman et al., 2014) with
maintained effects over a 6-month follow-up period (Kenardy, McCafferty, & Rosa,
2006). Less work has evaluated whether webCBT is effective for preventing anxiety
symptoms, and these findings are mixed (e.g. see Sánchez-Gutiérrez, Barbeito, & Calvo,
2017; Deady et al., 2017, for reviews). Experts are calling for additional, randomized-
controlled trials (RCTs) that select for individuals without mental health disorders at
baseline, to (a) understand the preventative potential of internet-based CBT and to (b)
identify the populations who may benefit most from webCBT prevention tools (see
Ebert, Cuijpers, Muñoz, & Baumeister, 2017, for a review). It is currently unknown
whether webCBT can serve as an effective early intervention to prevent anxiety symp-
tom escalation for medical and health science graduate students, although there is some
promising evidence that webCBT is efficacious for preventing suicidal ideation among
medical interns (Guille et al., 2015).
The primary aim of this study was to address gaps in the literature regarding
effective anxiety disorder prevention programs for medical and health science
graduate students, who are at significant risk for developing anxiety-related pro-
blems and who experience barriers to receiving face-to-face interventions. We
tested the efficacy of webCBT versus passive psychoeducation (i.e. web-based
mental health symptom questionnaires with automated feedback and referrals as
a control group [CG]) for anxiety symptom prevention among medical and health
science graduate students, using an RCT design. Data were assessed at baseline
(before the academic year) and reassessed after 3 months (during the academic
year). We hypothesized that (1) individuals assigned to the webCBT group would
report lower anxiety symptoms than individuals in the CG at follow-up assess-
ment. We also hypothesized that a smaller proportion of individuals in the
webCBT group, versus CG group, would: (2) meet a cutoff score suggestive of
clinically increased anxiety symptoms and/or (3) would not demonstrate a clini-
cally significant increase in symptoms during the school year (regardless of clinical
status).
388 A. N. HOWELL ET AL.

Methods
Participants
Please see Figure 1. In total, 1,507 graduate students from the College of Medicine,
College of Dental Medicine, College of Graduate Studies, College of Health Professions,
College of Nursing, and College of Pharmacy at a US medical university were sent an
e-mail 2 months prior to commencing the 2014–2015 academic year—inviting them to
participate in the study. Participants were eligible for the study if they were a returning
or new student beginning classes in August 2014 in one of the six colleges at the
university. E-mail invitations were returned as undeliverable for 11.9% (179/1,507) of
potential participants, and 71.0% (943) of individuals who received the e-mail (1,328)
agreed to participate in the study (webCBT: n = 468; CG: n = 475). E-mail addresses of
consenting students were assigned to the CG or MoodGYM conditions using the
method of 1:1 simple randomization using a random number generator. Of those
who agreed, 12.2% (58) assigned to the CG condition never initiated participation in

Figure 1. Participant flow chart following Consolidated Standards of Reporting Trials (CONSORT)
guidelines.
 COGNITIVE BEHAVIOUR THERAPY 389

the study and 24.1% (113) assigned to the webCBT condition never logged on to the
MoodGYM website; non-initiators reported slightly higher anxiety symptoms at base-
line (M = 6.18, SD = 5.26; per the GAD-7, see Measures, below) than initiators
(M = 5.25, SD = 4.83), t = 2.17, p = .03, and did not significantly differ on demographic
variables or depression, ps ≥ .10. The remaining 772 consenting individuals initiated
their allocated interventions (webCBT: n = 355; CG: n = 417).
Participants were screened at baseline and included in the current analyses if they
reported minimal to mild anxiety symptoms (see Measures, below), to test for the
prevention of anxiety symptoms. As a result, 17.4% (134/772) of consenting and
initiating participants were excluded from the current analyses due to moderate,
clinically increased anxiety at baseline (i.e. GAD-7 score ≥ 10, see also Measures,
below). The rate (17.4%) of clinically increased anxiety in the current sample is
consistent with prior work, which found that the point prevalence rate of a positive
screen for clinically significant anxiety (GAD-7 ≥ 10) was over eightfold higher in
medical students, residents, and fellows (19%) than in age-matched controls (2%;
Mousa et al., 2016) or in the general population (4%; Löwe et al., 2008). Also excluded
from analyses were 5.7% of participants (44/772) who initiated their interventions but
did not complete the baseline anxiety assessment. An intent-to-treat approach was used
to control for treatment effect biases for missing follow-up assessment data, such that
baseline measurements for non-completers were carried forward. No significant differ-
ences in baseline anxiety or depression emerged between eligible follow-up completers
and participants who did not complete the follow-up assessments, ts ≤ │.79│, ps ≥ .43.
There was a significantly greater percentage of follow-up assessment completers in the
CG (86.9%) versus webCBT (75.2%) condition,χ2 = 13.43, p < .001. The final sample
comprised 594 participants (webCBT: n = 266; CG: n = 328).
Assessment data were collected through a secure online website designed to maintain
confidentiality, with participant data identified only by a non-decodable number. All
participants were given information about symptoms and encouraged to seek treatment
locally if necessary. Participants received $20 prior to the start of the academic year for
completion of the baseline survey and $20 during the academic year for completion of
the follow-up survey in online gift certificates for compensation. A waiver of written
and oral consent, as well as study approval, was granted by the Institutional Review
Board at the Medical University of South Carolina.

Measures
Baseline assessment
Participants completed a secure baseline survey online, 6–8 weeks prior to starting the
academic year. The survey included demographic characteristics (e.g. gender, race/
ethnicity), type of academic program/college, and symptom measures.
The 7-Item Generalized Anxiety Disorder Scale (GAD-7; Spitzer, Kroenke, Williams,
& Löwe, 2006) is a measure of generalized anxiety disorder symptoms. It is also
moderately good at screening for panic, social anxiety, and posttraumatic stress dis-
orders (Kroenke, Spitzer, Williams, Monahan, & Löwe, 2007). Participants are asked
how often they have been bothered by anxiety in the past two weeks (e.g. trouble
relaxing). Participants respond on a 4-point scale, from 0 (not at all) to 3 (nearly every
390 A. N. HOWELL ET AL.

day). In general populations, confirmatory factor analyses substantiated a one-factor

structure, with factor invariance for gender and age (Löwe et al., 2008). In addition,
good internal consistency (α = .89) has been demonstrated, as well as criterion,
factorial, procedural, and construct validity in both general and clinical samples
(Kroenke et al., 2007; Löwe et al., 2008; Spitzer et al., 2006). Clinically relevant cutoff
scores have been psychometrically supported (Kroenke et al., 2007; Löwe et al., 2008;
Spitzer et al., 2006): (5 = Mild), (10 = Moderate and may warrant treatment),
(15 = Severe), and participants who already expressed clinically increased anxiety at
baseline (≥ 10) were excluded from current analyses. This cut-off score of 10 was also
used to operationalize whether participants did or did not experience clinically
increased anxiety symptoms at follow-up assessment. The percentage students who
met criteria for statistically reliable change (Jacobson & Truax, 1991) on the GAD-7 was
calculated using the GAD-7 baseline standard deviation and a test–retest reliability
coefficient of 0.83, as reported by Spitzer et al. (2006). The reliable change index (RCI)
in this sample = 3.03. Participants were categorized according to whether their anxiety
symptoms clinically significantly (a) improved, (b) remained unchanged, or deteriorated
from baseline to follow-up. In the current sample, the GAD-7 demonstrated good
internal consistency at both baseline (α = .76) and follow-up (α = .88) assessments.
The Patient Health Questionnaire (PHQ-9; Spitzer, Kroenke, & Williams, 1999) is a self-
report questionnaire assessing the frequency of nine DSM-IV depression items (e.g. “feeling
down, depressed, or hopeless”) in the past two weeks, with response options of 0 (not at all) to
3 (nearly every day). The PHQ-9 is commonly used for screening and informing diagnosis, as
well as selecting and monitoring treatment. The PHQ-9 is highly correlated with diagnosis by
mental health professionals and other depression assessment tools in a variety of populations
(Henkel et al., 2004; Kroenke, Spitzer, & Williams, 2001; Löwe, Kroenke, Herzog, & Gräfe,
2004; Martin, Rief, Klaiberg, & Braehler, 2006). In addition, the PHQ-9 has demonstrated
good to excellent internal consistency in both clinical and general/cigarette-smoking samples
(αs ≥ .86; Cannon et al., 2007; Löwe et al., 2004). While the PHQ-9 has historically been
moderately to strongly related to the GAD-7 (e.g. Spitzer et al., 2006), factor analysis has
confirmed two distinct dimensions between the PHQ-9 and the GAD-7 (Spitzer et al., 2006).
In the current sample, the PHQ-9 demonstrated good internal consistency, α = .80, and was
moderately related to the GAD-7 at both baseline and follow-up assessments (including both
intent-to-treat and per-protocol data), rs = .43—.51, ps < .001—suggesting that PHQ-9 scores
were unsurprisingly related to, but not duplicative of, GAD-7 scores. To test multilevel chi-
square tests (see Data Analytic Plan, below), the PHQ-9 was categorically recoded as “above”
or “below” a clinical cut-off score suggestive of depression pathology (i.e. scores ≥ 10;
Kroenke et al., 2001).

Follow-up assessment
Approximately, 3 months after the start of the academic year, participants were asked to
complete another secure web-based survey including current measures of anxiety
(GAD-7). They were also asked to report face-to-face mental health treatment history
since baseline assessment. Specifically, participants were asked to endorse or deny the
following statements: “I have not had any mental health problems”; “I have had some
mental health problems, but I have not sought help”; “I have consulted with [a mental
health provider/a general practitioner]”; and “I have been admitted to a psychiatric
 COGNITIVE BEHAVIOUR THERAPY 391

hospital”. Responses were recoded into “No psychological problems”; “Problems: Did
not seek help”; and “Problems: Sought help”.

Study procedures
Approximately, 2 months prior to starting the academic year, consenting participant
e-mail addresses were randomly assigned to either the webCBT or CG condition, using
complete simple randomization with equal allocation, via an online random sequence
generator operated by a person independent of the research. Participants in each group
were directed to the relevant websites via e-mail, including three weekly reminder
e-mails. Each e-mail provided information about the prevalence of mental health
problems including depression, anxiety, substance use, and suicide among graduate
students, as well as described symptoms of these problems. Students were encouraged
to seek in-person mental health treatment, if necessary. Contact information for urgent
and non-emergent local, confidential, and free mental health services was included in
each e-mail.

Control group
Following randomization, participants assigned to the CG were directed via e-mail each
week for four weeks to the institution’s Counseling and Psychological Services’ online
resource center (http://screening.mentalhealthscreening.org/musc) to complete an
anonymous mental health self-assessment related to mood, anxiety, and substance
use. Participants gained access to the website using their student ID and password
known only to the user. Upon completion of each set of symptom questionnaires,
which took approximately 10 min to complete (totaling about 40 min across four
weeks), participants were given automated clinical feedback about their scores. For
example, for anxiety, participants were told whether or not their screening was sugges-
tive of a specific anxiety disorder, although it was specified that screening was not a
substitute for a clinical evaluation and could not provide an actual diagnosis. Regardless
of the screening results, all students were encouraged to seek professional help if they
had concerns about emotional health or substance use and were provided with
resources.

WebCBT intervention
Participants assigned to the webCBT intervention were directed via e-mail each week
for four weeks to the intervention website (http://moodgym.anu.edu.au) to complete
one of four webCBT modules. Participants gained access to the secure website via a
username provided within the e-mail. Once participants accessed the website, they
created a unique password known only by them—allowing the content provided within
the modules (e.g. individualized CBT exercises) to remain anonymous. Website devel-
opers were able to track date and time of program access as a proxy for verification of
initiation in webCBT activities based on the assigned username. Due to study-related
restrictions, frequency of logins, duration spent per module, and degree of engagement
with interactive material could not be determined.
The webCBT program, MoodGYM, was developed by staff at the National
Institute for Mental Health Research at The Australian National University, to
392 A. N. HOWELL ET AL.

obviate barriers to CBT and to treat and prevent depression (Christensen, Griffiths,
& Korten, 2002). Since MoodGYM’s debut, several RCTs have been conducted to
determine its efficacy and effectiveness at treating depression and other disorder
symptoms, including anxiety (Twomey & O’Reilly, 2017, for a meta-analysis). There
are also promising results that MoodGYM may help to prevent both depression and
anxiety symptoms in adolescent and youth samples (e.g. Calear & Christensen,
2010), although less, and mixed yet promising, prevention work has been conducted
in adult populations (e.g. Deady et al., 2017; Powell et al., 2013). The program
consisted of 4 weekly web-based sessions lasting approximately 30 min each. The
interactive program uses exercises, quizzes, and scenarios to facilitate an under-
standing of the interplay between thoughts, emotions, and behaviors (Module 1)
and teaches cognitive restructuring techniques that promote the ability to identify
and challenge inaccurate, unrealistic, or overly negative thinking (Module 2 and 3).
The program also includes problem-solving strategies (Module 4).

Data analytic plan

Analyses were performed using SPSS, Version 24 (IBM corp., Armonk, NY). We first
assessed the distribution of demographic and baseline symptom severity according to
our randomization procedures. Specifically, we compared the webCBT and CG groups
on demographic and baseline clinical characteristics, using chi-square tests for catego-
rical variables and independent samples t-tests or analyses of variance (ANOVAs) for
continuous variables. A Bonferroni correction (α = .05/5 = .01) was used to control for
family-wise error rate for these preliminary analyses.
To test treatment efficacy, we compared pre-post changes between treatment condi-
tions on anxiety severity, clinical status, and reliable clinical change. A repeated
measures (RM) ANCOVA was used to test for differences in anxiety severity from
baseline to follow-up according to treatment condition, while co-varying for variables
with unequal allocation across groups. Effect size for baseline to follow-up score change
was assessed via Cohen’s d test, using marginal means (i.e. after covariation). An a
priori power analysis indicated that a total sample size of 200 (with n = 100 per
webCBT and CG group) was needed to detect a small (f = .10) significant within-
and between-subjects interaction effect. We explored whether baseline anxiety level (i.e.
minimal versus mild, per GAD-7 cut-off scores) moderated results, to inform who may
benefit the most from preventative webCBT. The RM ANCOVA test was then repeated,
but after adding covariates of gender and depression, which are known a priori to
impact anxiety and treatment outcomes (DiMatteo, Lepper, & Croghan, 2000; McLean
& Anderson, 2009; Pine, Cohen, Gurley, Brook, & Ma, 1998), to increase confidence
about the unique effect of treatment condition on follow-up anxiety symptoms. To test
clinical significance, clinical status and reliable clinical change were evaluated using the
follow-up data that was carried forward. Two-way and three-way chi square tests were
used to examine whether there were group differences in the proportion of individuals
who met a cut-off score for likely anxiety pathology (i.e. GAD-7 ≥ 10), as well as for
group differences in the proportion of individuals who had clinically significant symp-
tom change (i.e. improved, deteriorated, or remained unchanged), and if these propor-
tions differed across levels of our covariates or minimal/mild baseline anxiety status.
 COGNITIVE BEHAVIOUR THERAPY 393

Results
Demographic and clinical characteristics
Please see Table 1. Demographic variables, academic college, and baseline depressive
and anxiety symptoms were approximately equally distributed between conditions, with
the exception of race/ethnicity; there was a higher percentage of minority students in
the CG. Race/ethnicity groups did not significantly vary in follow-up anxiety symptoms,
p = .68. Treatment-seeking during the study also differed by condition; about three
times the number of participants in the webCBT group who experienced a psycholo-
gical problem endorsed seeking face-to-face mental health treatment (n = 28), versus
not seeking treatment (n = 9), at follow-up assessment. However, this proportion was
equal (ns = 24) for participants with psychological problems in the CG. Treatment-
seeking groups did not impact symptom change within each condition, ps > .70, but
significantly varied in follow-up anxiety symptoms, p < .001. Therefore, race and mental
health treatment-seeking were included as covariates for all primary analyses. Both a
priori variables, gender (t = 2.83, p = .005) and baseline depression (r = .37, p < .001),
were significantly related to follow-up anxiety.

Table 1. Demographic and clinical characteristics of medical and health science graduate students
during the 2014–2015 academic year.
webCBT Feedback Only (CG) Group Differences
N 266 328 —
Agea 28.0(8.1) 27.0(6.1) t = 1.43
d = 0.14
Genderb 70% female 66% female χ2 = 1.05
φ = 0.04
Race/Ethnicity c
84% White 75% White χ2 = 10.05*
6% Black 7% Black φ = 0.13
6% Asian 7% Asian
.4% Middle Eastern .6% Middle Eastern
.4% Multiracial 3% Multiracial
.4% Other 3% Other
3% Latino/a 4% Latino/a
Academic College 27% Health Professions 31% Health Professions χ2 = 6.32
23% M.D. 28% M.D. φ = 0.10
14% Nursing 13% Nursing
13% Pharmacy 9% Pharmacy
13% Graduate Studies 13% Graduate Studies
6% Dental 10% Dental
Mental Health Treatment-Seekingd 82% Denied Problems 84% Denied Problems χ2 = 6.30*
4% Problems: No Tx 8% Problems: No Tx φ = 0.11
14% Problems: Tx 8% Problems: Tx
Baseline Anxiety 3.28(2.63) 3.41(2.71) t = 0.62
Range: 0–9 Range: 0–9 d = 0.05
Follow-Up Anxietye 2.97(3.59) 3.62(3.67) t = 2.31*
Range: 0–16 Range: 0–21 d = 0.18
Baseline Depression 3.96(3.28) 3.61(3.48) t = 1.31
Range: 0–17 Range: 0–27 d = 0.10
Note: *p < .05; all other ps ≥ .10
Abbreviations: webCBT = web-based cognitive behavioral therapy; CG = control group; M.D. = medical
resident; Tx = (professional mental health) treatment
a
n = 429 due to missing data
b
n = 587 due to missing data
c
Latino/a ethnicity is non-exclusive; to meet cell count assumptions for chi-square analysis, race/ethnicity was recoded
into White, Black, Asian, and Other.
d
Treatment-seeking since baseline assessment; n = 501 due to missing data
e
Baseline data carried forward for missing follow-up data
394 A. N. HOWELL ET AL.

Treatment efficacy
Anxiety severity
See Table 2 and Figure 2. There was a small, significant multivariate interaction effect
between time (Baseline; Follow-up) and treatment condition (webCBT; CG) when
predicting change in anxiety symptom scores and while accounting for race and
treatment-seeking.1,2 There was also a small, significant between-subjects effect for
treatment condition, collapsed across time and accounting for covariates. The covaried
treatment effect (i.e. using marginal means) for anxiety symptoms at follow-up assess-
ment was small, t = 2.65, p = .008, d = .24, CI(95)d = .06—.42; students who engaged in
webCBT prior to the academic year reported less anxiety at follow-up (marginal M
[SD] = 2.88[3.36]; CI[95%] = 2.42–3.34) than students assigned to the CG (marginal M
[SD] = 3.69[3.35]; CI[95%] = 3.30–4.07).3
Interaction and between-subjects effects were robust and small-to-moderate when
additionally covarying for gender and baseline depression, F(472) = 4.44, η2 p = .01,

Table 2. Treatment efficacy of webCBT versus a control group on anxiety symptoms for medical and
health science graduate students during the 2014–2015 academic year.
F η2 p p
Multivariate Effects
Time (Baseline vs Follow-up) 5.74 .01 .02
Time*Race/Ethnicity .76 .002 .39
Time*Treatment-Seeking 17.29 .03 < .001
Time*Treatment Condition 4.93 .01 .03
Between-Subjects Effects (averaged across assessment time-points)
Race/Ethnicity 1.09 .002 .30
Treatment-Seeking 77.95 .14 < .001
Treatment Condition 4.17 .01 .04
Note: Bold font indicates significant probability results (p < .05)

Figure 2. Significant time-by-assignment interaction effect, while controlling for race/ethnicity and
mental health treatment-seeking. GAD-7 = Generalized Anxiety Disorder-7; webCBT = web-based
cognitive behavioral therapy; CG = control group.
Note: GAD-7 scores 5–9 = mildly increased anxiety; error bars = standard error; marginal means t = 2.65, p = .008,
d = .24.
 COGNITIVE BEHAVIOUR THERAPY 395

p = .04; d = .29, CI(95)d = .11—.48. We then included baseline anxiety cutoff score
status (i.e. minimal versus mild) as a between-subjects variable in the model. (Of note,
there were no significant differences in condition assignment between the minimal and
mild anxiety groups, χ2 [1] = .17, p = .69.) There was a significant three-way interaction,
F (1, 495) = 4.06, p = .04, for time, condition, and baseline anxiety status. The covaried
treatment effect was significant only for mildly anxious students and was of moderate
size, t = 3.08, p = .002, d = .48, CI(95)d = .17—.79, Mdiff = 1.53 (minimally anxious:
Mdiff = .39).

Clinical status
We then tested group differences in the proportion of individuals who met the clinical
cutoff score for anxiety pathology at follow-up. See Table 3. Results indicated that 4.51%
(n = 12) of webCBT participants developed clinically increased anxiety, compared with
8.54% (n = 28) of CG participants, χ2 (1) = 3.79, p = .05; LR = 3.92, p = .048; φ = .05.4
There was about a 47% reduction in the risk of developing clinically increased anxiety
symptoms for webCBT, versus CG participants, RR = .53, CI(95) = .27–1.02. The small
effect for the differences in distribution was stronger among participants with mild
(φ = .12) versus minimal (φ = .06) anxiety at baseline. The number needed to treat
(NNT) to prevent one minimally or mildly anxious student from developing a clinically
increased score during the school year (follow-up) was 25. The NNT for only mildly
anxious students, however, was 12.
We then explored whether chi-square results varied by our control variables of race/
ethnicity and treatment-seeking, as well as gender and depression. Results did not vary by
significance status according to race/ethnicity, baseline depression cutoff status (see Measures
section), or help-seeking status, all ps > .09. However, there was a difference for gender.5 The
results were significant for women (CG: n = 23, 10.8% [within CG]; webCBT: n = 10, 5.4%
[within webCBT]), χ2 (1) = 3.84, p = .05; LR = 3.96, p = .046; φ = .10, RR = .50, but not for men
(CG: n = 5, 4.5%; webCBT: n = 2, 2.5%), χ2 (1) = .55, p = .46; LR = .57, p = .45; φ = .05,
RR = .55). However after switching the order of chi-square levels, for full perspective, results
revealed gender gaps that trended toward significance for CG participants, χ2 (1) = 3.23,
p = .057; LR = 4.00, p = .045; φ = .11, RR = 2.39, but not for webCBT participants, χ2 (1) = 1.09,
p = .30; LR = 1.21, p = .27; φ = .06, RR = 2.16. A sensitivity power analysis indicated that a
small (but larger) effect size (φ = .14) would have been required for power = .80, α = .05, and
for the subsample size of men in the current study (n = 191), compared with φ = .10 for the
subsample of women (n = 403).
Using a hierarchical binary logistic regression analysis, we further explored potential
predictors of clinical status at follow-up, while covarying for proximity to clinical status
at baseline (i.e. baseline GAD-7 scores). When controlling for baseline anxiety as a
continuous variable (baseline anxiety: Wald = 34.18, df = 1, p < .001, B = .40), the odds
of reporting clinically increased anxiety at follow-up were 51% lower in the webCBT
group compared with the CG, albeit to a near-significant degree, Wald = 3.33, df = 1,
p = .068, exp[B] = .51, CI(95) = .25–1.05. The effect of treatment group was significant,
however, Wald = 4.13, p = .04, Exp(B) = .46, when additionally covarying for gender
(p = .21), baseline depression (p = .045), and race/ethnicity (composite and contrast ps
≥ .60), all Walds ≤ 4.02, but not when adding help-seeking (composite p = .01, contrast
ps ≥ .15) to the model, p = .18, Exp[B] = .59.
396 A. N. HOWELL ET AL.

Reliable clinical change

See Table 3. Based on our criteria for statistically reliable change, 16.5% (n = 54)
demonstrated reliable improvement in the CG, and 18.8% (n = 50) demonstrated
reliable improvement in the webCBT group. Conversely, 18.6% (n = 61) demonstrated
reliable deterioration in the CG, whereas 12.4% (n = 33) demonstrated reliable dete-
rioration in the webCBT group. An approximately equal percentage of students did not
have a clinically significant change in symptoms between the webCBT (70.0%; n = 140)
and CG (72.6%; n = 207) conditions. Thus, when conducting chi-square analyses to
determine proportional differences in change directionality (see Table 3), we only
assessed students who demonstrated a clinically significant change; these results neared
significance, both χ2 s and LRs = 3.42, ps ≥ .07, φs = 13. Proportional differences were
greater among participants with mild baseline anxiety, χ2 (1) = 4.46, p = .04; LR = 3.60,
φ = .20 (minimal: p = .96, φ = .01).6

Discussion
Experts have increasingly called for psychological resiliency-building programs for
medical and health science graduate students (Beresin et al., 2016; Rakesh et al.,
2017). In response to these calls, and to our knowledge, the current study is the first
to test the efficacy of webCBT versus a CG (i.e. automated symptom questionnaire
feedback) for preventing anxiety symptoms within this population.

Anxiety severity
Anxiety symptom severity at follow-up was significantly lower (albeit by about one
point) and to a small effect for students who were assigned to webCBT, compared with
the CG, and this effect was robust to race/ethnicity, in-person treatment-seeking,

Table 3. Rates of possible anxiety disorder status and directions of reliable clinical change per
treatment condition and anxiety levels at baseline.
CG (n = 266) webCBT (n = 328)
Baseline Anxiety Status (GAD-7 scores)
Clinical Status at Follow-Up (GAD-7 ≥ 10)
Reliable Change at Follow-Up N % n %
Minimal (n = 397)
GAD-7 ≥ 10 6 2.8 2 1.1
Improved 15 6.9 9 5.0
Unchanged 164 75.2 146 81.6
Deteriorated 39 17.9 24 13.4
Mild (n = 197)
GAD-7 ≥ 10 22 20.0 10 11.5
Improved 39 35.5 41 47.1
Unchanged 49 44.5 37 42.5
Deteriorated 22 20.0 9 10.0
Total (n = 594)
GAD-7 ≥ 10 28 8.5 12 4.5
Improved 54 16.5 50 18.8
Unchanged 213 64.9 183 68.8
Deteriorated 61 18.6 33 12.4
Note: GAD-7 = Generalized Anxiety Disorder Scale- 7 Item; CG = Control Group; webCBT = web-based cognitive
behavioral therapy (MoodGYM)
 COGNITIVE BEHAVIOUR THERAPY 397

gender, and baseline depression symptoms. This effect was significantly moderated by
baseline anxiety severity status (i.e. minimal versus mild), suggesting that students who
endorse mild anxiety (5 ≤ GAD-7 < 10) before the start of the school year may benefit
the most from webCBT prevention efforts, to a moderate statistical effect (i.e. by about
two points, consistent with prior MoodGYM anxiety treatment findings using the
GAD-7 in an unselected general population; Powell et al., 2013).

Clinical status and degree of reliable clinical change

About 4.5% of the low-anxious students who engaged in webCBT developed clinically
increased anxiety during the academic year, and this was a significantly lower propor-
tion than the 8.5% who developed clinically increased anxiety in the CG. The webCBT
rate for a positive GAD-7 screen (≥ 10) falls within the 95% CI (3.5–4.6) of the national
rate (Mousa et al., 2016), whereas the CG rate falls above this CI. Furthermore,
participants in the webCBT group were 46% less likely to report clinically increased
anxiety at follow-up than the CG, while covarying for gender, race/ethnicity, baseline
depression, and baseline anxiety. For the subsample of students who met mild anxiety
status before the academic year, clinical status rates at follow-up were 11.5% and 20%
for the webCBT and CG conditions, respectively. This (subsample) rate for positive
screens in the webCBT group was nearly half the rate of both the CG and a separate
sample of medical students, residents, and fellows (19%; Mousa et al., 2016). The NNT
was 25 (full sample) and 12 (mild anxiety subsample), consistent with a review and
meta-analysis of Internet-based interventions for the prevention of mental disorders
(NNT ≈ 9 to 41; Sander, Rausch, & Baumeister, 2016).
Only within the subsample of mildly anxious, higher-risk students was there was a
significant difference in the proportion of clinically significant improvement versus
deterioration across the webCBT (47% versus 10%) and CG (36% versus 20%) condi-
tions, regardless of clinical status. Thus, while webCBT did not fully prevent clinically
significant anxiety symptoms from developing during the academic year, results suggest
that webCBT may: (a) have a small to moderate preventative effect on anxiety escalation
for a population that is at high risk for anxiety problems; (b) be more indicated for
students who report mild, versus minimal, anxiety before the academic year; and (c)
reduce the discrepancy of possible anxiety disorder rates between general and medical/
graduate student populations.

Effect sizes
It is worth highlighting that our between-subjects and interaction effect sizes were small,
overall, but were moderate for the subgroup of students with mild baseline anxiety (Cohen,
1988). These small-to-moderate effect sizes are comparable to other webCBT prevention/
early intervention studies among undergraduates (Cukrowicz & Joiner, 2007; Musiat et al.,
2014) and in the general population (standardized mean difference of .31 for both Deady
et al., 2017; and Moreno-Peral et al., 2017; per meta-analytic reviews). Small corrective
treatment effect sizes have similarly been found for MoodGYM-specific interventions for
depression and anxiety in prior work (e.g. Twomey & O’Reilly, 2017). The lack of
significant findings in the current study for students with minimal anxiety at baseline is
398 A. N. HOWELL ET AL.

consistent with non-significant post-study effects of webCBT on anxiety-related symptoms

for healthy/low anxious participants from general, undergraduate, and injury patient
populations (Christensen et al., 2014; Mouthaan et al., 2013; Musiat et al., 2014; Deady
et al., 2017; for a meta-analytic review).
It warrants consideration that the type of comparison group may have also had some
impact on the size of our treatment effect. For example, a meta-analysis (Conley et al.,
2017) of indicated mental health prevention programs for any targeted problem (e.g.
anxiety, depression, social skills) for at-risk (i.e. subclinical) undergraduate and grad-
uate students found that the effect of anxiety-specific interventions at follow-up assess-
ment was larger when the intervention was compared with a no intervention or wait-list
CG versus an attention CG. Moreno-Peral et al. (2017) also found larger meta-analytic
effects of anxiety prevention interventions with wait-list CGs.
With regard to therapist involvement, the obtained effect sizes parallel meta-
analytic results for webCBT treatment for clinically increased anxiety symptoms;
web-based interventions with therapist support demonstrated a large effect size
(d = 1.00) on anxiety symptom severity, whereas interventions without therapist
support, such as in the current study, demonstrated a small effect size (d = .24;
Palmqvist, Carlbring, & Andersson, 2007; Spek et al., 2007; Twomey & O’Reilly,
2017). Thus, it is possible that the effect sizes obtained in the current study would
have been larger if we had compared outcomes from the self-guided, webCBT
intervention group to a more typical scenario of no proactive psychoeducation or
feedback (i.e. wait-list or no intervention); yet, it is important to capitalize on any
opportunity to raise awareness about mental health problems and services in a high-
risk population. Areas for effect size improvement may also lie in the addition of
therapist support, when feasible.

Considerations for gender and depression

In-line with prior work (DiMatteo et al., 2000; McLean & Anderson, 2009; Pine et al.,
1998), we found that gender and baseline depression were directly related to anxiety
severity at follow-up. The treatment effect on clinical status was stronger for women
than for men, and the gender gap in clinical status at follow-up was greater for the CG
condition than for the webCBT group. The development of anxiety disorders, therefore,
may especially be a concern for female students, compared with male students; it
logically follows that any preventative effects of webCBT on clinical status may be
greater for women than for men. Gender did not, however, significantly impact the
interaction between time and treatment condition on anxiety severity—suggesting that
gender differences in treatment outcomes are likely more apparent among clinically
severe samples. It is also worth mentioning that men comprised about 33% of the total
sample (consistent with our institution’s demographic characteristics). Given the lower
base rate of anxiety disorders among men versus women (e.g. McLean & Anderson,
2009), replication of this study with a greater sampling of male students may detect
small treatment effects for clinical status among men. Future studies expanding this
work may determine whether there is a need for gender-specific assessments or inter-
ventions, or if there is a potential benefit of screening for depression prior to the
academic year.
 COGNITIVE BEHAVIOUR THERAPY 399

Secondary outcomes: treatment-seeking

Among participants who endorsed having mental health problems during the course of
the study, significantly more participants in the webCBT group sought face-to-face
mental health treatment than participants in the CG. Based on data that were collected,
it is unclear whether the effect of treatment-seeking on follow-up anxiety is subsumed
by a circumstantial allocation of treatment-seekers to treatment condition, or if expo-
sure to webCBT increased rates of face-to-face treatment. Existing data suggest that
webCBT (MoodGYM) may boost treatment-seeking for face-to-face CBT, compared
with an attention CG (Christensen, Leach, Barney, Mackinnon, & Griffiths, 2006), but a
higher percentage of consenting students did not initiate MoodGYM (24%) compared
with the CG (12%), and non-initiators in general reported higher anxiety at baseline
than initiators (i.e. by about 1 point, on average, on the GAD-7). The current findings
are notable, because there is a well-documented issue of mental health treatment
underutilization among medical and health science graduate students (Dunn et al.,
2008; Guille et al., 2015). Additional research may assess factors that modulate webCBT
adherence, such as motivation and willingness for intervention (Farrer, Griffiths,
Christensen, Mackinnon, & Batterham, 2014), gender and setting (Neil, Batterham,
Christensen, Bennett, & Griffiths, 2009), and perceived mental health stigma, as well as
the mechanisms by which webCBT may enhance treatment-seeking. For example, it
could be that webCBT improves attitudes and comfort about face-to-face treatment,
especially for populations faced with treatment disparities and unique barriers.

Limitations and future directions

In light of the present findings, there are limitations and additional future directions that
warrant consideration. First, the study was conducted at a single medical university, and
results may not generalize to other institutions. Results need to be replicated, within and
across institutions, to demonstrate valid preventative effects. Second, anxiety symptoms
were self-reported and could not be validated by standardized clinical assessments.
Although clinician-administered measures may ostensibly improve the validity of symptom
reporting, medical students are more likely to report more severe symptoms when mental
health assessments are anonymous compared with when they are only confidential (Levine,
Breitkopf, Sierles, & Camp, 2003). Third, it is important to highlight that while we could
track whether or not participants initiated webCBT, the number of webCBT module
completions, frequency of logins, duration of module use, and degree of interactive material
engagement could not be verified. In studies with low adherence (i.e. below 50% of the
modules), the meta-analytic treatment effect of MoodGYM on depression and anxiety has
been small (Twomey & O’Reilly, 2017). Future studies should incorporate methods of
confirming and/or controlling for module engagement, as well as for testing whether the
degree of engagement is influenced by gender, depression, race/ethnicity, or treatment-
seeking behavior. Fourth, tasks for the control (CG) versus webCBT groups differed in
attentional demand; the webCBT materials were more engaging and time-demanding than
the CG materials. The CG was designed to leverage a pre-existing assessment feature at the
university that is available at no cost to students. Future work would benefit from including
an attention CG into an RCT design, to rule out potentially confounding effects of
400 A. N. HOWELL ET AL.

engagement. Fifth, the present study did not assess for longer term effects of webCBT on
anxiety symptoms in this population. Future studies would benefit from longer and more
frequent follow-up periods, such as into the following academic year, to determine whether
effects are maintained after one course of webCBT or if students need “booster” modules of
webCBT before each academic year. Medical and graduate student stress levels are known
to fluctuate with time (Adams, 2004); thus, future work may identify when the dispersion of
webCBT would be optimally effective for students at risk for acute and/or chronic anxiety.
Current data provide additional justification for developing studies with longer follow-up
periods to address this question.
WebCBT may not only prevent the escalation of anxiety symptoms but also impact
areas of functioning, such as financial burden and withdrawal rates for students
struggling with graduate school stress. Another limitation of the current study, there-
fore, is that it lacks a measure of functioning that describes the impact of anxiety
symptoms (and/or symptom changes) on students’ lives. Future studies may determine
whether webCBT improves academic performance and generalizes to other important
areas associated with resiliency and quality of life (e.g. interpersonal relationships,
emotion regulation skills) for medical and health science graduate students.

Notes
1. The pattern of results held when not including covariates, F (1, 592) = 3.91, η2 p = .01, p = .048.
2. These results were consistent when analyses were conducted only among follow-up
assessment completers (i.e. per-protocol analysis), multivariate F (1, 481) = 4.80, p = .03.
3. The pattern of results held when not including covariates, (M[SD] = 2.97[3.16]) versus CG
(M[SD] = 3.62[3.67]) condition, t(598.83) = 2.31, p = .02, d = .19, CI(95)d = .03—.35.
Cohen’s d was corrected for dependence between the means, using Morris and DeShon’s
(2002) equation 8.
4. Results neared significance and the pattern of results held among follow-up assessment
completers (i.e. per-protocol analysis), χ2 (1) = 2.72, p = .13; φ = .07.
5. Gender did not moderate the strength of the time-treatment condition interaction found
in the RM ANCOVA results, F(1,471) = .33, η2 p = .001, p =.57.
6. Results were consistent when analyses were conducted only among follow-up assessment
completers (i.e. per-protocol analysis), χ2 (1) = 6.21, p = .045; LR = 6.27, p = .04, φ = .19
(minimal: p = .78, φ = .78).

Acknowledgments
The authors would like to acknowledge and thank the students for taking part in this study. The
authors would also like to thank and acknowledge Helen Christensen, PhD., Professor at the Black
Dog Institute, University of New South Wales, Sydney, Australia, who generously provided access to
MoodGym to allow the conduct of this study. The authors would also like to acknowledge the
assistance of Ms. Kylie Bennett, e hub manager, and Mr Anthony Bennett, software engineer, at the
Centre for Mental Health Research. Dr Christensen, Ms. Bennett, and Mr. Bennett provided no
financial support and were not involved in the design, analysis or interpretation of the study results.

Disclosure statement
No potential conflict of interest was reported by the authors.
 COGNITIVE BEHAVIOUR THERAPY 401

Funding
This work was supported by the Department of Health and Human Services (DHHS) and the
Substance Abuse and Mental Health Services Administration (SAMHSA) Garrett Lee Smith
Memorial Act, under Grant [1U79SM060490-01]; and the National Institute on Drug Abuse
(NIDA) under Grant [1K23DA039318-01]; and the National Institute of Mental Health (NIMH)
under Grant [T32MH018869-30].

ORCID
Ashley N. Howell http://orcid.org/0000-0002-9499-4688
Alyssa A. Rheingold http://orcid.org/0000-0003-3515-7529
Thomas W. Uhde http://orcid.org/0000-0002-4197-9695
Constance Guille http://orcid.org/0000-0001-6004-3027

References
Adams, J. (2004). Straining to describe and tackle stress in medical students. Medical Education,
38(5), 463–464. doi:10.1111/j.1365-2929.2004.01810.x
Adelman, C. B., Panza, K. E., Bartley, C. A., Bontempo, A., & Bloch, M. H. (2014). A meta-
analysis of computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety
disorders. Journal of Clinical Psychiatry, 75, 695–704. doi:10.4088/JCP.13r08894
Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014). Guided internet-
based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A
systematic review and meta-analysis. World Psychiatry, 13(3), 288–295. doi:10.1002/wps.20151
Beresin, E. V., Milligan, T. A., Balon, R., Coverdale, J. H., Louie, A. K., & Roberts, L. W. (2016).
Physician wellbeing: A critical deficiency in resilience education and training. Academic
Psychiatry, 40, 9–12. doi:10.1007/s40596-015-0475-x
Brimstone, R., Thistlethwaite, J. E., & Quirk, F. (2007). Behaviour of medical students in seeking
mental and physical health care: Exploration and comparison with psychology students.
Medical Education, 41, 74–83. doi:10.1111/j.1365-2929.2006.02649.x
Calear, A. L., & Christensen, H. (2010). Review of internet-based prevention and treatment
programs for anxiety and depression in children and adolescents. Medical Journal of Australia,
192(11), S12–S14.
Cannon, D. S., Tiffany, S. T., Coon, H., Scholand, M. B., McMahon, W. M., & Leppert, M. F.
(2007). The PHQ-9 as a brief assessment of lifetime major depression. Psychological
Assessment, 19, 247–251. doi:10.1037/1040-3590.19.2.247
Chew-Graham, C. A., Rogers, A., & Yassin, N. (2003). ‘I wouldn’t want it on my CV or their
records’: Medical students’ experiences of help-seeking for mental health problems. Medical
Education, 37, 873–880.
Christensen, H., Batterham, P., Mackinnon, A., Griffiths, K. M., Hehir, K. K., Kenardy, J., . . .
Bennett, K. (2014). Prevention of generalized anxiety disorder using a web intervention, iChill:
Randomized controlled trial. Journal of Medical Internet Research, 16(9), e199. doi:10.2196/
jmir.3507
Christensen, H., Griffiths, K. M., & Korten, A. (2002). Web-based cognitive behavior therapy:
Analysis of site usage and changes in depression and anxiety scores. Journal of Medical
Internet Research, 4(1), e3. doi:10.2196/jmir.4.1.e3
Christensen, H., Leach, L. S., Barney, L., Mackinnon, A. J., & Griffiths, K. M. (2006). The effect of
web based depression interventions on self-reported help seeking: Randomised controlled trial
[ISRCTN77824516]. BMC Psychiatry, 6(4), 1–11. doi:10.1186/1471-244X-6-1
402 A. N. HOWELL ET AL.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, MI:
Erlbaum.
Conley, C. S., Shapiro, J. B., Kirsch, A. C., & Durlak, J. A. (2017). A meta-analysis of indicated
mental health prevention programs for at-risk higher education students. Journal of
Counseling Psychology, 64(2), 121–140. doi:10.1037/cou0000190
Cuijpers, P., Marks, I. M., van Straten, A., Cavanagh, K., Gega, L., & Andersson, G. (2009).
Computer-aided psychotherapy for anxiety disorders: A meta-analytic review. Cognitive
Behaviour Therapy, 38(2), 66–82. doi:10.1080/16506070802694776
Cukrowicz, K. C., & Joiner, T. E. (2007). Computer-based intervention for anxious and depres-
sive symptoms in a non-clinical population. Cognitive Therapy and Research, 31(5), 677–693.
doi:10.1007/s10608-006-9094-x
Deady, M., Choi, I., Calvo, R. A., Glozier, N., Christensen, H., & Harvey, S. B. (2017). eHealth
interventions for the prevention of depression and anxiety in the general population: A
systematic review and meta-analysis. BMC Psychiatry, 17, 310. doi:10.1186/s12888-017-1489-6
DiMatteo, M. R., Lepper, H. S., & Croghan, T. W. (2000). Depression is a risk factor for
noncompliance with medical treatment: Meta-analysis of the effects of anxiety and depression
on patient adherence. Archives of Internal Medicine, 160(14), 2101–2107. doi:10.1001/
archinte.160.14.2101
Dunn, L. B., Iglewicz, A., & Moutier, C. (2008). A conceptual model of medical student well-
being: Promoting resilience and preventing burnout. Academic Psychiatry, 32, 44–53.
doi:10.1176/appi.ap.32.1.44
Dyrbye, L. N., Thomas, M. R., Huntington, J. L., Lawson, K. L., Novotny, P. J., Sloan, J. A., &
Shanafelt, T. D. (2006). Personal life events and medical student burnout: A multicenter study.
Academic Medicine, 81(4), 374–384. doi:10.1097/00001888-200604000-00010
Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression,
anxiety, and other indicators of psychological distress among US and Canadian medical
students. Academic Medicine, 81(4), 354–373. doi:10.1097/00001888-200604000-00009
Ebert, D. D., Cuijpers, P., Muñoz, R. F., & Baumeister, H. (2017). Prevention of mental health
disorders using internet-and mobile-based interventions: A narrative review and recommen-
dations for future research. Frontiers in Psychiatry, 8, 116. doi:10.3389/fpsyt.2017.00297
Enns, M. W., Cox, B. J., Sareen, J., & Freeman, P. (2001). Adaptive and maladaptive perfection-
ism in medical students: A longitudinal investigation. Medical Education, 35(11), 1034–1042.
Farrer, L. M., Griffiths, K. M., Christensen, H., Mackinnon, A. J., & Batterham, P. J. (2014).
Predictors of adherence and outcome in internet-based cognitive behavior therapy delivered in
a telephone counseling setting. Cognitive Therapy and Research, 38(3), 358–367. doi:10.1007/
s10608-013-9589-1
Griffiths, K. M., Farrer, L., & Christensen, H. (2010). The efficacy of internet interventions for
depression and anxiety disorders: A review of randomised controlled trials. Medical Journal of
Australia, 192(11), S4–S11.
Guille, C., Zhao, Z., Krystal, J., Nichols, B., Brady, K., & Sen, S. (2015). Web-based cognitive behavioral
therapy intervention for the prevention of suicidal ideation in medical interns: A randomized clinical
trial. JAMA Psychiatry, 72(12), 1192–1198. doi:10.1001/jamapsychiatry.2015.1880
Gulliver, A., Griffiths, K. M., & Christensen, H. (2010). Perceived barriers and facilitators to
mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10, 113.
doi:10.1186/1471-244X-10-113
Henkel, V., Mergl, R., Kohnen, R., Allgaier, A. K., Möller, H. J., & Hegerl, U. (2004). Use of brief
depression screening tools in primary care: Consideration of heterogeneity in performance in
different patient groups. General Hospital Psychiatry, 26(3), 190–198. doi:10.1016/j.
genhosppsych.2004.04.005
Hyun, J. K., Quinn, B. C., Madon, T., & Lustig, S. (2006). Graduate student mental health: Needs
assessment and utilization of counseling services. Journal of College Student Development, 47
(3), 247–266. doi:10.1353/csd.2006.0030
 COGNITIVE BEHAVIOUR THERAPY 403

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining
meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology,
59, 12–19. doi:10.1037/0022-006X.59.1.12
Kenardy, J., McCafferty, K., & Rosa, V. (2006). Internet-delivered indicated prevention for
anxiety disorders: Six-month follow-up. Clinical Psychologist, 10, 39–42. doi:10.1080/
13284200500378746
Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9. Journal of General Internal
Medicine, 16(9), 606–613. doi:10.1046/j.1525-1497.2001.016009606.x
Kroenke, K., Spitzer, R. L., Williams, J. B., Monahan, P. O., & Löwe, B. (2007). Anxiety disorders
in primary care: Prevalence, impairment, comorbidity, and detection. Annals of Internal
Medicine, 146(5), 317–325. doi:10.7326/0003-4819-146-5-200703060-00004
Levine, R. E., Breitkopf, C. R., Sierles, F. S., & Camp, G. (2003). Complications associated with
surveying medical student depression. Academic Psychiatry, 27(1), 12–18. doi:10.1176/appi.
ap.27.1.12
Löwe, B., Decker, O., Müller, S., Brähler, E., Schellberg, D., Herzog, W., & Herzberg, P. Y. (2008).
Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the
general population. Medical Care, 46(3), 266–274. doi:10.1097/MLR.0b013e31817c6100
Löwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004). Measuring depression outcome with a
brief self-report instrument: Sensitivity to change of the Patient Health Questionnaire (PHQ-
9). Journal of Affective Disorders, 81, 61–66. doi:10.1016/S0165-0327(03)00171-X
Martin, A., Rief, W., Klaiberg, A., & Braehler, E. (2006). Validity of the brief patient health
questionnaire mood scale (PHQ-9) in the general population. General Hospital Psychiatry, 28,
71–77. doi:10.1016/j.genhosppsych.2005.07.003
Mazzola, J. J., Walker, E. J., Shockley, K. M., & Spector, P. E. (2011). Examining stress in
graduate assistants: Combining qualitative and quantitative survey methods. Journal of
Mixed Methods Research, 5(3), 198–211. doi:10.1177/1558689811402086
McLean, C. P., & Anderson, E. R. (2009). Brave men and timid women? A review of the gender
differences in fear and anxiety. Clinical Psychology Review, 29(6), 496–505. doi:10.1016/j.
cpr.2009.05.003
Moreno-Peral, P., Conejo-Cerón, S., Rubio-Valera, M., Fernández, A., Navas-Campaña, D.,
Rodríguez-Morejón, A., . . . Rodríguez-Bayón, A. (2017). Effectiveness of psychological and/or
educational interventions in the prevention of anxiety: A systematic review, meta-analysis, and
meta-regression. JAMA Psychiatry, 74(10), 1021–1029. doi:10.1001/jamapsychiatry.2017.2509
Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with
repeated measures and independent-groups designs. Psychological Methods, 7, 105–125. doi:
10.1037//1082-989X.7.1.105
Mousa, O. Y., Dhamoon, M. S., Lander, S., & Dhamoon, A. S. (2016). The MD blues: Under-
recognized depression and anxiety in medical trainees. PloS one, 11(6), e0156554. doi:10.1371/
journal.pone.0156554
Mouthaan, J., Sijbrandij, M., De Vries, G. J., Reitsma, J. B., Van de Schoot, R., Goslings, J. C., . . .
Olff, M. (2013). Internet-based early intervention to prevent posttraumatic stress disorder in
injury patients: Randomized controlled trial. Journal of Medical Internet Research, 15(8), e:165.
doi:10.2196/jmir.2460
Musiat, P., Conrod, P., Treasure, J., Tylee, A., Williams, C., & Schmidt, U. (2014). Targeted
prevention of common mental health disorders in university students: Randomised controlled
trial of a transdiagnostic trait-focused web-based intervention. PLoS One, 9(4), e93621.
doi:10.1371/journal.pone.0093621
Myers, S. B., Sweeney, A. C., Popick, V., Wesley, K., Bordfeld, A., & Fingerhut, R. (2012). Self-
care practices and perceived stress levels among psychology graduate students. Training and
Education in Professional Psychology, 6, 55–66. doi:10.1037/a0026534
Neil, A. L., Batterham, P., Christensen, H., Bennett, K., & Griffiths, K. M. (2009). Predictors of
adherence by adolescents to a cognitive behavior therapy website in school and community-
based settings. Journal of Medical Internet Research, 11(1), e6. doi:10.2196/jmir.1050
404 A. N. HOWELL ET AL.

Novotney, A. (2014). Students under pressure. Monitor on Psychology, 45. Retrieved from http://
www.apa.org/monitor/2014/09/cover-pressure.aspx
Oswalt, S. B., & Riddock, C. C. (2007). What to do about being overwhelmed: Graduate students,
stress and university services. College Student Affairs Journal, 27, 24–44.
Palmqvist, B., Carlbring, P., & Andersson, G. (2007). Internet-delivered treatments with or
without therapist input: Does the therapist factor have implications for efficacy and cost?
Expert Review of Pharmacoeconomics & Outcomes Research, 7(3), 291–297. doi:10.1586/
14737167.7.3.291
Pine, D. S., Cohen, P., Gurley, D., Brook, J., & Ma, Y. (1998). The risk for early-adulthood
anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Archives
of General Psychiatry, 55, 56–64. doi:10.1001/archpsyc.55.1.56
Powell, J., Hamborg, T., Stallard, N., Burls, A., McSorley, J., Bennett, K., . . . Christensen, H.
(2013). Effectiveness of a web-based cognitive-behavioral tool to improve mental well-being in
the general population: Randomized controlled trial. Journal of Medical Internet Research, 15,
e2. doi:10.2196/jmir.2240
Rakesh, G., Pier, K., & Costales, T. L. (2017). A call for action: Cultivating resilience in healthcare
providers. American Journal of Psychiatry Residents’ Journal, 12(4), 3–5. doi:10.1176/appi.ajp-
rj.2017.120402
Reavley, N., & Jorm, A. F. (2010). Prevention and early intervention to improve mental health in
higher education students: A review. Early Intervention in Psychiatry, 4(2), 132–142.
doi:10.1111/(ISSN)1751-7893
Roberge, P., Fournier, L., Duhoux, A., Nguyen, C. T., & Smolders, M. (2011). Mental health
service use and treatment adequacy for anxiety disorders in Canada. Social Psychiatry and
Psychiatric Epidemiology, 46(4), 321–330. doi:10.1007/s00127-009-0179-1
Sánchez-Gutiérrez, T., Barbeito, S., & Calvo, A. (2017). A revision of preventive web-based
psychotherapies in subjects at risk of mental disorders. International Journal of Interactive
Multimedia & Artificial Intelligence, 4(5), 50–54. doi:10.9781/ijimai.2017.459
Sander, L., Rausch, L., & Baumeister, H. (2016). Effectiveness of internet-based interventions for
the prevention of mental disorders: A systematic review and meta-analysis. JMIR Mental
Health, 3(3), e38. doi:10.2196/mental.6061
Sareen, J., Jagdeo, A., Cox, B. J., Clara, I., Ten Have, M., Belik, S. L., . . . Stein, M. B. (2007).
Perceived barriers to mental health service utilization in the United States, Ontario, and the
Netherlands. Psychiatric Services, 58(3), 357–364. doi:10.1176/ps.2007.58.9.1181
Spek, V., Cuijpers, P. I. M., Nyklíček, I., Riper, H., Keyzer, J., & Pop, V. (2007). Internet-based
cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis.
Psychological Medicine, 37(3), 319–328. doi:10.1017/S0033291706008944
Spitzer, R. L., Kroenke, K., Williams, J. B., & Patient Health Questionnaire Primary Care Study
Group. (1999). Validation and utility of a self-report version of PRIME-MD The PHQ primary
care study. JAMA, 282(18), 1737–1744. doi:10.1001/jama.282.18.1737
Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for assessing
generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097.
doi:10.1001/archinte.166.17.1863
Stecker, T. (2004). Well-being in an academic environment. Medical Education, 38(5), 465–478.
doi: 10.1046/j.1365-2929.2004.01812.x
Thompson, A., Hunt, C., & Issakidis, C. (2004). Why wait? Reasons for delay and prompts to
seek help for mental health problems in an Australian clinical sample. Social Psychiatry and
Psychiatric Epidemiology, 39(10), 810–817. doi:10.1007/s00127-004-0816-7
Tjia, J., Givens, J. L., & Shea, J. A. (2005). Factors associated with undertreatment of medical
student depression. Journal of American College Health, 53(5), 219–224. doi:10.3200/
JACH.53.5.219-224
Twomey, C., & O’Reilly, G. (2017). Effectiveness of a freely available computerised cognitive
behavioural therapy programme (MoodGYM) for depression: Meta-analysis. Australian &
New Zealand Journal of Psychiatry, 51(3), 260–269. doi:10.1177/0004867416656258
 COGNITIVE BEHAVIOUR THERAPY 405

Tyssen, R., Dolatowski, F. C., Røvik, J. O., Thorkildsen, R. F., Ekeberg, Ø., Hem, E., . . .
Vaglum, P. (2007). Personality traits and types predict medical school stress: A six-year
longitudinal and nationwide study. Medical Education, 41(8), 781–787. doi:10.1111/j.1365-
2923.2007.02802.x
Wang, P. S., Lane, M., Olfson, M., Pincus, H. A., Wells, K. B., & Kessler, R. C. (2005). Twelve-
month use of mental health services in the United States: Results from the national comor-
bidity survey replication. Archives of General Psychiatry, 62(6), 629–640. doi:10.1001/
archpsyc.62.6.629