Suicide and Life-Threatening Behavior 1

© 2019 The American Association of Suicidology

DOI: 10.1111/sltb.12554

Safety Planning on Crisis Lines: Feasibility,

Acceptability, and Perceived Helpfulness of a
Brief Intervention to Mitigate Future Suicide
Risk
CHRISTA D. LABOULIERE, PHD, BARBARA STANLEY, PHD, ALISON M. LAKE, MA, AND
MADELYN S. GOULD, MPH, PHD

Background: The role of crisis hotlines traditionally was limited to de-escalation

and service linkage. However, hotlines are increasingly recruited to provide
outreach and follow-up to suicidal individuals. Hotlines have the opportunity to
not just defuse current crises but also provide brief interventions to mitigate
future risk. The Safety Planning Intervention (SPI) is a brief intervention
designed to help manage suicidal crises, but its feasibility and effectiveness on
hotlines are not established.
Aims: This study examined feasibility and perceived effectiveness of SPI, as
reported by 271 crisis counselors at five centers in the National Suicide Prevention
Lifeline network.
Method: Counselors were trained to use SPI. Self-report surveys were completed
immediately after training (time 1) and at the end of the study, approximately
9 months later (time 2).
Results: Counselors reported that SPI was feasible and helpful, and was used on
both incoming and follow-up calls. Utilization and perceived effectiveness at time
2 were predicted by self-efficacy, feasibility, and helpfulness at time 1.
Limitations: Results are preliminary and limited to counselors’ perceptions. Future
RCTs should establish efficacy of SPI for crisis callers.
Conclusion: The Safety Planning Intervention is a promising approach to reduce
crisis callers’ future suicide risk that hotline counselors report is both feasible and
helpful.

Each year, more than 44,000 individuals in component of suicide prevention efforts in
the United States die by suicide (Center for the United States since the mid-1960s
Disease Control and Prevention, 2016). Cri- (Litman, Farberow, Shneidman, Heilig, &
sis hotlines have become an integral Kramer, 1967). Historically, crisis hotlines

CHRISTA D. LABOULIERE, BARBARA STANLEY,
AND MADELYN S. GOULD, Department of
Psychiatry, Columbia University, New York, NY,
USA and New York State Psychiatric Institute,
New York, NY, USA; ALISON M. LAKE, New York
State Psychiatric Institute, New York, NY, USA.
This project was funded by the Substance
Abuse and Mental Health Services Administration
(SAMHSA) through a subcontract from ICF Macro,
Inc (PI: M. Gould) and the American Foundation
for Suicide Prevention (PI: B. Stanley).
Address correspondence to Christa D.
Labouliere, PhD, Department of Psychiatry,
Columbia University, 1051 Riverside Drive, Unit
100, New York, NY 10032; E-mail: Christa.
Labouliere@nyspi.columbia.edu
2 SAFETY PLANNING ON CRISIS LINES
have been widely available in most metropoli-
tan areas, regardless of individuals’ financial
or transportation restrictions (Dew, Bromet,
Brent, & Greenhouse, 1987); however, poli-
cies, access, and the quality of services
received varied widely between communities.
In January 2005, the National Suicide
Prevention Lifeline (Lifeline; 1-800-273-
TALK) was created, connecting a network of
more than 160 crisis centers across the coun-
try (Lifeline, 2018). The Lifeline assures that
callers in crisis can call this toll-free number,
24 hr a day, from any location in the United
States and be connected to a trained local
counselor who can provide risk assessment,
de-escalation, crisis intervention, and linkage
to mental health services (Joiner et al., 2007).
This access can save lives, as suicidal individu-
als often eschew more formal help-seeking
(Deane, Wilson, & Ciarrochi, 2001; Labou-
liere, Kleinman, & Gould, 2015), and thus
may not have supports in place when a suici-
dal crisis occurs. The Lifeline is effective at
both de-escalating suicidal crises (Gould,
Kalafat, HarrisMunfakh, & Kleinman, 2007)
and increasing mental health service utiliza-
tion (Gould, Munfakh, Kleinman, & Lake,
2012).
While the role of crisis hotlines has
been limited traditionally to crisis de-escala-
tion and service linkage, their role has begun
to expand. More recently, crisis hotlines have
been utilized increasingly to provide follow-
up care to suicidal individuals after discharge
from an emergency department or inpatient
psychiatric facility for an acute suicidal crisis
or following a suicide crisis call. Post dis-
charge follow-up contact shows preliminary
evidence of reducing subsequent suicidal
behavior (Gould et al., 2018; Luxton, June, &
Comtois, 2013). Given this success, crisis hot-
lines have begun to expand beyond crisis
de-escalation and started to employ brief psy-
chotherapeutic interventions to reduce future
suicide risk (Arias, Sullivan, Miller, Camargo,
& Boudreaux, 2015; Boudreaux et al., 2013;
Gould, Cross, Pisani, Munfakh, & Kleinman,
2013; Stanley et al., 2015, 2016). These brief
interventions typically provide support and
psychoeducation, assist with means
reduction, and teach coping skills that can be
used to weather a future suicidal crisis (Che-
sin et al., 2017; Fleischmann et al., 2008;
Johnson, Frank, Ciocca, & Barber, 2011).
A promising brief intervention to
reduce future suicide risk is the Safety Plan-
ning Intervention (SPI; Stanley & Brown,
2012). SPI has its roots in cognitive behav-
ior therapy shown to reduce repetition of
suicide attempts over an 18-month interval
(Brown et al., 2005). Identified by the Sui-
cide Prevention Resource Center as a best
practice for suicide prevention (SPRC,
2018), SPI can be administered as a stand-
alone intervention delivered by trained pro-
fessionals or paraprofessionals in a single
session taking 20–45 min (Stanley & Brown,
2012; Stanley et al., 2015, 2016). During
SPI, the provider imparts psychoeducation
about the fluctuating nature of suicide risk,
the suicidal individual conveys the narrative
of his or her crisis so that crisis warning
signs can be identified, and together, they
produce a written, prioritized list of specific
coping strategies and sources of support
that could be used to alleviate a future suici-
dal crisis (Stanley & Brown, 2012). The
steps of the safety plan include the follow-
ing: recognizing warning signs and antece-
dents of a suicidal crisis; utilizing internal
coping strategies, social contacts and adap-
tive social settings to distract from suicidal
thoughts; seeking help from family mem-
bers or close friends; contacting mental
health professionals, agencies, or emergency
services; and restricting access to means.
Once the safety plan form has been com-
pleted, providers can assess the suicidal
individual’s likelihood of using the plan,
engage in strategies to enhance motivation,
and problem-solve potential obstacles.
Given that the literature supports
SPI’s effectiveness in a range of acute care
settings (Stanley & Brown, 2012; Stanley
et al., 2015, 2016), translation to suicide
hotline services settings appears viable and
indicated. However, the use of future-
oriented approaches or psychotherapeutic
interventions is relatively novel to crisis hot-
line counselors, some of whom are lay
LABOULIERE ET AL.
volunteers and may not have extensive train-
ing in mental health. Conducting SPI on
crisis hotlines requires additional training, as
well as a shift in orientation regarding hotli-
nes’ role from diffusing current suicidal
crises to preventing future ones (Allen, For-
ster, Zealberg, & Currier, 2002). Neverthe-
less, hotline counselors are well situated to
provide such interventions, given their expe-
rience interacting with and access to suicidal
individuals. However, no study to date has
explored whether SPI is feasible or effective
on crisis lines.
In this study, we assessed the feasibil-
ity and perceived effectiveness of SPI as
reported by Lifeline counselors. We hypoth-
esized that: (1) after completing training,
Lifeline counselors would report that SPI is
feasible during incoming crisis calls and fol-
low-up calls, thereby supporting utilization
of SPI in crisis call settings; (2) after train-
ing, Lifeline counselors would report a high
value of SPI training, high levels of per-
ceived helpfulness for the intervention, and
high levels of self-efficacy over their ability
to apply SPI to the crisis call setting, sug-
gesting that training in SPI is sufficient for
translation into the crisis call setting; and (3)
at
9 months post-training, Lifeline coun-
selors will report that they utilized SPI and
found it effective with both incoming crisis
calls and follow-up calls, thereby indicating
preliminary perceived effectiveness in the
crisis call setting.
METHOD
Participants
Six crisis centers in the Lifeline net-
work from across four U.S. census regions
were awarded a competitive grant from
SAMHSA in 2008 to develop a clinical fol-
low-up program for suicidal callers (Gould
et al., 2018). Average call volume at the six
centers was slightly under 7,000 calls per
month (range = 390–25,000), and the aver-
age number of personnel at each center
responsible for answering calls was 54.3
3
(range = 18–90). Three of the six centers used
only paid staff, while the other three used an
average of 70% volunteers to answer crisis
calls (range = 40%–90%). The percentage of
calls to each center that concerned suicide
ranged from 5% to 40% (average = 15.2%).
As part of the competitive application process,
each center was required to design its own fol-
low-up protocol; as a result, centers’ pro-
grams varied considerably according to
factors such as the number and timing of fol-
low-up calls offered and the number of crisis
counselors assigned to a particular case. How-
ever, SAMHSA specified that crisis coun-
selors at participating crisis centers must
receive SPI training and centers agreed to
pilot SPI on follow-up calls. The smallest of
the six centers did not contribute enough data
to be included in current analyses (i.e., n = 12
at time 1 and n = 0 at time 2), leaving a total
of five centers included in the study.
Participants were crisis counselors
from the five participating Lifeline centers.
All participants were responsible for handling
incoming crisis calls from suicidal callers, and
between 3 and 16 crisis counselors at each
center were also responsible for conducting
follow-up calls with suicidal callers to their
centers. Two-hundred and seventy-one Life-
line crisis counselors completed an evalua-
tion immediately after SPI training (time 1),
and 58% (N = 158) also completed an
evaluation at the end of the study, an average
of 9 months post-training (time 2; M =
8.73 months, SD = 4.46 months). Partici-
pants had worked as crisis counselors for
approximately 4 years at the time of SPI
training (M = 3.90 years, SD = 5.65 years),
and the sample was relatively evenly split
between volunteer (45%) and paid (55%)
staff members. Approximately 60% had a
graduate degree, 30% a bachelor’s degree,
and 10% a high school diploma only; no
other demographic data were available.
There were no significant differences
between individuals who did and did not
complete the time 2 evaluation (Years of
Experience: F2,43 = 0.18, p = .84; Volunteer
¼271Þ = 3.19, p = .20; Education
2
Status: Xð2;N
Level: Xð6;N ¼271Þ = 1.98, p = .92).
2
4 SAFETY PLANNING ON CRISIS LINES
Measures
Immediately following training, partic-
ipants completed a 20-item questionnaire
adapted from the Learning Transfer System
Inventory (LTSI; Holton, Bates, & Ruona,
2000; Holton, Chen, & Naquin, 2003) by the
evaluation team to meet the objectives of the
study (see Table 1); 19 quantitative items
were rated using a 5-point Likert scale where
higher values indicated greater agreement,
and one open-ended qualitative item assessed
barriers to SPI. Quantitative questions were
designed to assess perceptions of SPI after
training completion, including: (1) feasibility
of SPI (7 items), (2) helpfulness of SPI (3
items), (3) value of training (4 items), and (4)
confidence in one’s ability to competently use
SPI (i.e., self-efficacy; 5 items). Exploratory
factor analysis confirmed a four-factor solu-
tion consistent with the theoretical constructs
intended for measurement (see Data Analysis
section for details). All subscales displayed
good internal consistency (Feasibility: a = .85,
Helpfulness: a = .71, Value of Training:
a = .80, and Self-Efficacy: a = .84). At time 2,
participants completed a 5-item question-
naire created by the evaluation team to assess
utilization of SPI on crisis calls and follow-up
calls (2 items; a = .79), how effective they
found SPI on crisis calls and follow-up calls (2
items; a = .74), and a single open-ended qual-
itative item assessing barriers to SPI. Quanti-
tative items were rated using a 5-point Likert
scale ranging from 1 (Not at all) to 5 (Very).
Procedure
This study was conducted as a supple-
ment to a broader evaluation of the Lifeline; a
detailed description of the methods for that
evaluation has been provided in an associated
study (Gould et al., 2018). Six centers partici-
pated in the broader evaluation, but one cen-
ter was omitted from the current study due to
insufficient data (e.g., no data at time 2). The
study was approved by the Institutional
Review Board of (The New York State
Psychiatric Institute).
Between February 2010 and February
2011, all staff members assigned to conduct
follow-up calls at the five participating crisis
centers were required to be trained in SPI.
Additional staff members who were not
assigned the follow-up call function also
attended the training. Approximately half of
participants attended a 2-hr in-person train-
ing conducted by the co-creators of SPI
(53.7%), which was rolled out across the cen-
ters using a dynamic wait-list design, and
those who could not attend an in-person ses-
sion viewed a recording of the training held at
their facility (46.3%). Going forward, new
hires to the crisis center also viewed this
recording as part of their standard onboard-
ing protocol. Participant demographics,
responses, or attrition did not vary by
in-person versus recorded training modality
¼271Þ = 21.23, p = .73), so all subse-
2
(Xð26;N
quent analyses were collapsed across modal-
ity. At the end of training (time 1),
participants completed an anonymous evalua-
tion. At the end of the study, an average of
9 months later, participants were asked to
complete an additional anonymous survey
(time 2).
Data Analysis
All analyses were run in IBM SPSS Sta-
tistics for Macintosh, Version 24.0 (Armonk,
NY: IBM Corp). Because the scale in the
study was project-adapted, factor analysis was
used to determine the internal structure of
the measure and empirically corroborate sub-
scales. Before proceeding to factor analysis,
all data were screened for linearity, normality,
and homoscedasticity, and Kaiser-Meyer-
Olkin measures of sampling adequacy and
Bartlett’s test of sphericity were conducted to
make certain factor analysis was indicated
(Bryant & Yarnold, 1995; Garson, 2007).
Next, an exploratory principal axis factor
analysis of all item scores was conducted to
determine the least number of factors
accounting for the most common variance in
this set of variables. Principal axis factor ana-
lysis typically yields the same solution as the
LABOULIERE ET AL. 5

TABLE 1
Descriptive Statistics and Factor Structure of the Safety Planning Intervention Time 1 Survey
Self- Helpful- Feasibility Value of
efficacy ness of SPI of SPI training
Mean
Item (SD) C P S P S P S P S

I have sufficient training 4.04 (0.65) 0.67 0.89 0.81

to implement SPI
successfully
I feel well-prepared to use 4.10 (0.61) 0.59 0.67 0.76
SPI
I am positive about my 4.13 (0.67) 0.61 0.57 0.74
ability to implement SPI
with callers
I feel confident that I can 4.26 (0.57) 0.51 0.56 0.70
be an effective SPI crisis
worker
I feel comfortable with 4.20 (0.62) 0.43 0.47 0.62
SPI principles and
techniques
SPI does not offer 3.58 (0.92) 0.40 0.62 0.62
anything beyond what I
already use with callers
(r)
Our current training/way 3.28 (0.90) 0.45 0.58 0.64
is sufficient without
adding the SPI model (r)
SPI will be helpful on 4.28 (0.86) 0.61 0.48 0.63 0.33 0.58
crisis calls
A change in my priorities 3.79 (1.14) 0.57 0.67 0.70
is needed to implement
SPI during crisis calls (r)
Too much is happening to 3.96 (1.00) 0.65 0.64 0.76
adequately implement
SPI during crisis calls (r)
SPI is a unique 3.94 (1.00) 0.42 0.61 0.60
intervention on crisis
calls
I feel using SPI on crisis 4.09 (0.93) 0.58 0.33 0.48 0.61 0.70
calls is a burden (r)
Adopting SPI for use on 3.89 (0.86) 0.48 0.57 0.66
crisis calls will not be too
difficult
SPI procedures are 3.97 (0.80) 0.43 0.51 0.61
consistent with my
previous crisis training
The resources I need to 4.14 (0.80) 0.54 0.45 0.66
implement SPI will be
available to me

(continued)
6 SAFETY PLANNING ON CRISIS LINES

TABLE 1
(continued)

Self- Helpful- Feasibility Value of

efficacy ness of SPI of SPI training
Mean
Item (SD) C P S P S P S P S

The way SPI was taught 4.05 (0.75) 0.78 0.88 0.88
made me feel confident
that I could apply it
during crisis calls
It is clear that the SPI 3.90 (0.96) 0.57 0.60 0.73
trainers understand how
I will use what I learned
The SPI training helped 4.06 (0.69) 0.47 0.54 0.66
me know how to apply
my learning to crisis calls
Situations used in training 3.91 (0.86) 0.39 0.44 0.59
are similar to those I
encounter on crisis calls
Eigenvalues 7.22 1.46 0.96 0.51
% Variance 38.00 7.67 5.06 2.70
a .84 .71 .85 .80
Subscale Mean (SD) 4.14 (0.49) 3.75 (0.71) 3.96 (0.70) 4.00 (0.64)
Subscale Correlations
1 – 0.29*** 0.58*** 0.66***
2 – 0.48*** 0.46***
3 – 0.56***
4 –

N = 271. Range for all items is 1–5. (r) denotes items that were reverse-scored. All items loaded on
at least one factor; two items had a pattern coefficient > 0.32 on more than one factor, and they were
assigned to the highest loading factor to improve interpretability. Loadings less than 0.32 are not
presented, as they are unstable.
C, Communality; P, Pattern coefficient; S, Structure coefficient.
***p < .01.

more commonly used principal components
analysis, but is more appropriate for situa-
tions where the underlying latent constructs
are likely to covary and produces less inflated
estimates of variance accounted (Costello &
Osborne, 2005). The number of factors
retained was determined using parallel analy-
sis, which detects the number of factors with
eigenvalues greater than those expected by
chance from random permutations of the data
(Fabrigar, Wegener, MacCallum, & Strahan,
1999). An oblique rotation (oblimin with Kai-
ser normalization) was used to clarify the data
structure, as correlations between subscales
were anticipated (Costello & Osborne, 2005).
The pattern matrix was examined to deter-
mine which items were associated with each
factor, and items were selected for a factor if
they had a minimum loading of 0.32, repre-
senting
10% overlapping variance between
items (Tabachnick & Fidell, 2001). Subscale
scores were created by averaging all items
loading on a factor.
A single open-ended item assessing
barriers to use SPI was asked at both time 1
and time 2. Responses were coded for themes
to saturation by two independent trained
coders; a total of four categories were derived,
LABOULIERE ET AL.
including: (1) caller attitude, ability, or will-
ingness; (2) center constraints; (3) crisis coun-
selor attitude or willingness; and (4) other.
Interrater reliability for the coding system
was very high (intraclass coefficient of 0.980
at time 1 and 0.976 at time 2).
Analysis of variance was used to
determine time 1 group differences. Due
to the nested structure of the data (i.e.,
crisis counselors nested within centers),
multilevel modeling was used to identify
predictors of SPI utilization and perceived
effectiveness at time 2 (Raudenbush &
Bryk, 2002), accounting for nonindepen-
dence of observations. First, an uncondi-
tional model with no covariates was
estimated to determine variance explained
by site. Next, two multilevel random
effects models were run with SPI utiliza-
tion and perceived effectiveness at time 2
as dependent variables and time 1 feasibil-
ity, helpfulness, value of training, and self-
efficacy as level 1 independent variables.
Data were analyzed using linear mixed
models of the following form, where Out-
comeij represents time 2 SPI utilization or
perceived effectiveness as reported by crisis
worker i in center j:
Outcomeij ¼ b0j þ b1j Feasibilityij
þ b2j Helpfulnessij þ b3j Valueij
þ b4j Self-Efficacy ij þ eij
b0j ¼ c00 þ u0j
b1j ¼ c10 þ u1j
b2j ¼ c20 þ u2j
b3j ¼ c30 þ u3j
b4j ¼ c40 þ u4j :
Restricted maximum likelihood esti-
mation was used to estimate parameters,
and overall fit of the models was evaluated
by examining the Akaike information crite-
rion (Akaike, 1974) and likelihood ratio
test (Luke, 2004; Raudenbush & Bryk,
2002). All tests were subjected to a modi-
fied Bonferroni adjustment to correct for
multiple testing.
7
RESULTS
Internal Structure
Principal axis factor analysis using par-
allel analysis and oblimin rotation with Kaiser
normalization produced a four-factor solu-
tion, corroborating the measure’s four sub-
scales (see Table 1). Communality estimates
ranged from 0.39 to 0.78, indicating an ade-
quate level of reliability. The four factors
accounted for
53% of the variance and
showed intercorrelations, validating the
choice of an oblique rotation. The pattern
matrix was examined to determine which
items were associated with each factor, and
items were selected for a factor if they had a
minimum loading of 0.32. All items loaded on
at least one factor; two items had a pattern
coefficient > 0.32 on more than one factor,
and they were assigned to the highest loading
factor to improve interpretability and reduce
measurement error. Examination of the
structure matrix supported item assignment,
as each item showed a high correlation with
its associated factor.
Response to SPI Immediately Post-
Training
Descriptive statistics are presented in
Table 2. Immediately after completing train-
ing, crisis counselors reported that SPI would
be both feasible (M = 3.96, SD = 0.70) and
helpful (M = 3.75, SD = 0.71) on crisis calls.
Crisis counselors reported a high value of
training (M = 4.00, SD = 0.64) and a high
level of self-efficacy for using SPI on future
crisis calls (M = 4.14, SD = 0.49). There
were significant differences in response across
sites regarding perceptions of feasibil-
ity (F4,268 = 47.55, p < .001), helpfulness
(F4,272 = 13.66, p < .001), value of train-
ing (F4,270 = 11.13, p < .001), and self-effi-
cacy (F4,270 = 6.73, p < .001), in that
counselors at one crisis center (A) reported
consistently lower scores than the other four
crisis centers (B–E). When responses to the
qualitative question about barriers to SPI
were examined, 88% of counselors at crisis
8 SAFETY PLANNING ON CRISIS LINES

center A stated that their call center served

6.73***

5.06***
47.55***
13.66***
11.13***

10.16***

All values are Mean (SD) unless otherwise denoted. N = 271 at time 1 and N = 158 at time 2. Site A had significantly lower scores than all other sites at
F more of a triage function, with linkage to ser-
vices as their primary responsibility. This call
center required incoming calls to be limited
in length (e.g., typically ~7 min) and focus
n = 81 (29.9%)

n = 41 (25.9%)
only on immediate response to the current
4.07 (0.54)
3.84 (0.61)
4.05 (0.63)
4.15 (0.46)

3.30 (1.04)
4.18 (1.00)
Site E

crisis including emergency rescue if needed;

with these constraints, it would be very chal-
lenging, if not impossible, to implement SPI.
The other four call centers (B–E) were more
positive regarding barriers to SPI, with
n = 24 (8.9%)

n = 13 (8.2%)

37.5% of crisis counselors reporting no barri-

4.23 (0.59)
3.90 (0.83)
4.13 (0.59)
4.23 (0.45)

3.73 (0.48)
4.31 (0.90)

ers at all; 24.3% reporting concerns about

Site D

caller attitudes, ability, or willingness; and

23.4% reporting crisis counselor attitudes or
willingness as the primary barrier.
Descriptive Statistics of the Safety Planning Intervention Immediately Post-Training and 9 months Later

Utilization and Perceived Helpfulness of

n = 43 (15.8%)

n = 30 (19.0%)
3.90 (0.51)
3.71 (0.59)
4.14 (0.53)
4.21 (0.44)

3.82 (0.86)
4.24 (0.94)

SPI at 9 months Post-Training

Site C

Descriptive statistics are presented in

Table 2. Although SPI was originally
expected to be used primarily on follow-up
calls, crisis counselors reported that they had
n = 87 (32.1%)

n = 57 (36.1%)

utilized SPI with suicidal callers on both

4.26 (0.45)
3.95 (0.63)
4.09 (0.53)
3.77 (0.57)

3.41 (1.17)
4.32 (0.85)

incoming crisis calls (M = 3.39, SD = 1.08)

Site B

and follow-up calls (M = 3.50, SD = 1.08)

over the 9-month post-training period. SPI
was perceived to be effective on both types of
calls (incoming: M = 4.11, SD = 1.07; fol-
low-up: M = 4.13, SD = 1.08). As no differ-
n = 36 (13.3%)

n = 17 (10.8%)

ences were detected in utilization (t52 = 0.80,

2.87 (0.69)
3.05 (0.71)
3.39 (0.73)
4.23 (0.50)

2.50 (0.94)
2.74 (1.09)
Site A

p = .43) or effectiveness (t50 = 1.02, p = .31)

between incoming crisis calls and follow-up
calls, the two utilization and effectiveness
questions were averaged.
As at time 1, there were significant dif-
3.96 (0.70)
3.75 (0.71)
4.00 (0.64)
4.14 (0.49)

3.39 (1.06)
4.09 (1.04)

ferences in responses across site at the time 2

N = 271

N = 158
All sites

evaluation. The same site that had reported

large logistical barriers to SPI at time 1 (Site
A) reported significantly lower utilization
(F4,153 = 5.06, p < .001) and perceived effec-
Perceived effectiveness

tiveness (F4,154 = 10.16, p < .001) of SPI at

both time 1 and time 2.
Post-training (time 1)

time 2 than all other sites (B–E). When

At 9 months (time 2)
Value of training

***p < .001.

responses to the qualitative question about

barriers to SPI were examined, 73.3% of crisis
Self-efficacy
Helpfulness

Utilization
Feasibility

counselors at Site A reported that center con-

TABLE 2

straints had limited their use of SPI. In con-

trast, at Sites B–E, 52.9% of crisis counselors
Item

reported no barriers to using, SPI at time 2;

LABOULIERE ET AL.
25.7% cited caller attitudes, ability, or will-
ingness as the primary barrier; 5.7% reported
crisis counselor attitudes or willingness; and
only 2.3% reported center constraints.
Predictors of SPI Utilization and Perceived
Effectiveness
The unconditional models (without
any covariates) examined variance accounted
for by each level (counselors and centers) for
SPI utilization and effectiveness as reported
at time 2. The majority of variance in SPI uti-
lization was accounted for by between-worker
variation (83.7%), with the clustering of crisis
counselors within centers accounting for
16.3%. The influence of center was more pro-
nounced for perceived effectiveness, where
clustering accounted for 31.8% of the vari-
ance and 68.2% was accounted for by
between-worker variation. Next, multilevel
models with level 1 covariates (time 1 report
of feasibility, helpfulness, value of training,
self-efficacy) were estimated; examination of
fit indices suggested adequate model fit supe-
rior to the unconditional models. Higher
time 1 self-efficacy was associated with
greater utilization of SPI at time 2. Time 1
perceptions of feasibility and helpfulness pre-
dicted reports of SPI effectiveness at time 2
(see Table 3).
DISCUSSION
In this study, we assessed the feasibility
and perceived effectiveness of SPI as reported
by crisis hotline counselors at five crisis cen-
ters in the Lifeline network. Immediately
after completing training, crisis counselors
reported that SPI would be both feasible and
helpful on crisis calls, stated that training in
SPI was valuable, and endorsed a high level of
self-efficacy for using SPI on future crisis
calls. Over a 9-month post-training period,
crisis counselors reported that they had uti-
lized SPI with suicidal callers on both
incoming crisis calls and follow-up calls and
that SPI had been effective on both types of
calls. Higher self-efficacy immediately after
9
training was associated with greater utiliza-
tion of SPI over the 9-month post-training
period, and perceptions of feasibility and
helpfulness immediately after the training
predicted reports of SPI effectiveness approx-
imately 9 months later. These results suggest
that SPI is translatable to the crisis center set-
ting, and may be a promising intervention for
reducing callers’ future suicide risk. These
findings add to the larger body of work
demonstrating that SPI can be readily incor-
porated across a wide range of acute care set-
tings and administered by both mental health
and lay providers with appropriate training
(Stanley & Brown, 2012; Stanley et al., 2015,
2016).
However, while SPI may be a useful
brief intervention for some crisis centers, our
data also suggest that certain center con-
straints may limit its effectiveness. Call cen-
ters may have greater difficulty implementing
SPI if they experience extremely high call vol-
ume that severely limits the length of calls or
rely predominantly on the provision of refer-
rals rather than crisis counseling. Previous
reports have examined the importance of
shifting staff’s perception of their role and
workload in order to support safety planning
(Allen et al., 2002; Chesin et al., 2017). Our
data corroborate that attitudes about SPI are
important to the intervention’s perceived
effectiveness, in that beliefs about SPI’s help-
fulness and feasibility immediately post-train-
ing predicted subsequent reports of the
intervention’s effectiveness over the subse-
quent 9-month period. However, our data
suggest that other factors beyond attitudes
also play an important role. Our findings sug-
gest that the likelihood of clinical uptake after
training is also heavily influenced by imple-
mentation climate; leadership at call centers
wishing to implement SPI will need to
address center constraints that conflict with
successful utilization of SPI (e.g., short call
times, emphasis on current crisis). While
studies examining both implementation and
effectiveness during dissemination trials are
becoming more common in other disciplines
(Cucciare et al., 2016; Damschroder et al.,
2017; Zatzick et al., 2015), this approach has
10 SAFETY PLANNING ON CRISIS LINES

TABLE 3
Multilevel Modeling of Predictors of SPI Utilization and Perceived Helpfulness at 9-Months
Post-Training
Utilization of SPI

Unconditional model
Estimate (SE) t p CI (99%)
Intercept (c00) 3.36 (0.22) 15.27 .001*** 2.23 to 4.48
Variance components Estimate (SE) z p CI (99%)
Residual (eij) 1.03 (0.12) 8.73 .001*** 0.77 to 1.38
Intercept (u0j) 0.20 (0.18) 1.10 .27 0.02 to 2.08
Multilevel model
Estimate (SE) t p CI (99%)
Intercept (c00) 0.09 (0.86) 0.10 .92 2.34 to 2.16
Feasibility (c10) 0.17 (0.17) 0.98 .33 0.29 to 0.63
Helpfulness (c20) 0.12 (0.15) 0.85 .40 0.26 to 0.50
Value of training (c30) 0.69 (0.21) 0.34 .74 0.47 to 0.61
Self-efficacy (c40) 0.48 (0.25) 1.93 .04* 0.17 to 1.13
Model Fit AIC 441.62
2LL 445.62

Perceived effectiveness of SPI

Unconditional model
Estimate (SE) t p CI (99%)
Intercept (c00) 3.96 (0.30) 13.28 .001*** 2.54 to 5.40
Variance components Estimate (SE) z p CI (99%)
Residual (eij) 0.88 (0.10) 8.65 .001*** 0.65 to 1.18
Intercept (u0j) 0.41 (0.32) 1.27 .21 0.05 to 3.14
Multilevel model
Estimate (SE) t p CI (99%)
Intercept (c00) 0.61 (0.71) 0.85 .40 2.47 to 1.26
Feasibility (c10) 0.36 (0.14) 2.49 .02* 0.03 to 0.75
Helpfulness (c20) 0.47 (0.12) 3.86 .001*** 0.15 to 0.79
Value of training (c30) 0.23 (0.17) 1.34 .18 0.22 to 0.68
Self-efficacy (c40) 0.10 (0.21) 0.50 .62 0.44 to 0.65
Model Fit AIC 383.62
2LL 379.62

*p < .05; **p < .01; ***p < .001.

only begun to be applied to suicide preven-
tion (Stanley, 2017; Labouliere et al., 2018).
Future studies should further investigate what
differentiates settings that are and are not
conducive to SPI, and how these obstacles to
effective implementation can best be over-
come.
In addition to implementation factors,
our findings also acknowledge the importance
of enhancing providers’ self-efficacy to
improve the likelihood of new interventions’
clinical uptake. Research suggests that
infusing trainings with active learning strate-
gies and opportunities to practice new skills
can improve confidence in assisting at-risk
individuals (Cross, Matthieu, Lezine, &
Knox, 2010; Wyman et al., 2008), and that
increasing self-efficacy also increases the like-
lihood of using newly trained behaviors
(Ajzen, 1991; 2011). Despite the evidence of
self-efficacy’s importance to behavior change,
evaluations of suicide prevention trainings
rarely focus on this factor relatively to other
indicators of program success (Hangartner,
LABOULIERE ET AL.
Totura, Labouliere, Greglewicz, & Karver,
2019). Our findings corroborate the broader
literature that one’s self-efficacy over newly
trained behavior is a critical predictor of that
behavior’s subsequent use, suggesting that
suicide prevention trainings may be more suc-
cessful if building confidence and applied
skills training were actively integrated and
evaluated.
The current study has several limita-
tions that must be acknowledged, and thus,
results should be considered preliminary.
First, no control group was utilized, so it is
possible that utilization or perceived effec-
tiveness of SPI could have been influenced by
factors other than SPI training or crisis center
site. Further, selection bias may have played a
role. The Lifeline centers participating in our
study were not randomly selected; rather,
sites were selected for inclusion based on a
competitive application process, which could
mean their staff were better prepared to uti-
lize SPI than the average crisis center. As no
pre-training evaluation was able to be con-
ducted due to time constraints, it is possible
that participants may have already had posi-
tive attitudes or high self-efficacy prior to par-
ticipating in training. In addition, it is
possible that those who completed time 2 sur-
veys may have been more motivated to do so
if they had positive experiences using SPI, so
these findings may not be generalizable to all
crisis counselors. Lastly, effectiveness in this
REFERENCES
AJZEN, I. (1991). The theory of planned
behavior. Organizational Behavior and Human Deci-
sion Processes, 50, 179–211.
AJZEN, I. (2011). The theory of planned
behavior: Reactions and reflections. Psychology &
Health, 26, 1113–1127.
AKAIKE, H. (1974). A new look at the statis-
tical model identification. IEEE Transactions on
Automatic Control, 19, 716–723.
ALLEN, M. H., FORSTER, P., ZEALBERG, J., &
CURRIER, G. (2002). Report and recommendations
regarding psychiatric emergency and crisis services: A
review and model program descriptions. American
Psychiatric Association (APA) Task Force on Psy-
chiatric Emergency Services. Retrieved January 1,
11
study was limited to the perceptions of Life-
line crisis counselors; future studies should
also examine the effectiveness of SPI as
reported by crisis callers directly.
Despite these limitations, the results
of this study warrant further investigation
and suggest that SPI is a promising approach
to reduce crisis callers’ future suicide risk.
Our findings suggest that translation of SPI
to suicide hotline service settings is both
viable and indicated, providing an opportu-
nity for hotline counselors to expand their
role from crisis de-escalation to provision of
follow-up care and interventions to reduce
future suicide risk. Given that the period
after an acute crisis is an extremely high-risk
period for suicidal behavior, and that many
suicidal individuals never make contact with
mental health services, crisis hotlines may be
particularly well suited to provide such inter-
ventions. Further, our findings indicate that
crisis hotline counselors find SPI both feasi-
ble and helpful on both incoming crisis and
follow-up calls. Future studies should con-
duct randomly controlled trials of SPI with a
wider array of crisis centers to truly establish
the intervention’s effectiveness in the crisis
setting. Future studies should also examine
the role of implementation context (such as
center constraints) and counselor character-
istics (such as self-efficacy) when examining
the dissemination of interventions into new
settings.
2018, from www.psych.org/edu/otherres/libarc
hives/archives/tfr/tfr200201.pdf
ARIAS, S. A., SULLIVAN, A. F., MILLER, I.,
CAMARGO, C. A., JR,, & BOUDREAUX, E. D.
(2015). Implementation and use of a crisis hot-
line during the treatment as usual and universal
screening phases of a suicide intervention study.
Contemporary Clinical Trials, 45(Part B), 147–
150.
BOUDREAUX, E. D., MILLER, I., GOLDSTEIN,
A. B., SULLIVAN, A. F., ALLEN, M. H., MANTON, A.
P., ET AL. (2013). The emergency department
safety assessment and follow-up evaluation (ED-
SAFE): Method and design considerations. Con-
temporary Clinical Trials, 36, 14–24.
12
BROWN, G. K., TEN HAVE, T., HEN-
RIQUES, G. R., XIE, S. X., HOLLANDER, J. E., &
BECK, A. T. (2005). Cognitive therapy for the
prevention of suicide attempts: A randomized
control trial. Journal of the American Medical
Association, 294, 563–570.
BRYANT, F. B., & YARNOLD, P. R. (1995).
Principal-components analysis and exploratory
and confirmatory factor analysis. In L. G. Grimm,
& P. R. Yarnold (Eds.), Reading and understanding
multivariate statistics (4th ed., pp. 99–136).
Washington, DC: American Psychological
Association.
Center for Disease Control and Prevention,
National Center for Injury Prevention and Con-
trol. (2016). Web-based Injury Statistics Query and
Reporting System (WISQARS), 2016 statistics.
Retrieved January 1, 2018, from www.cdc.gov/in
jury/wisqars
CHESIN, M. S., STANLEY, B., HAIGH, E. A.,
CHAUDHURY, S. R., PONTOSKI, K., KNOX, K. L.,
ET AL. (2017). Staff views of an emergency depart-
ment intervention using safety planning and struc-
tured follow-up with suicidal veterans. Archives of
Suicide Research, 21, 127–137.
COSTELLO, A. B., & OSBORNE, J. W. (2005).
Best practices in exploratory factor analysis: Four
recommendations for getting the most from your
analysis. Practical Assessment, Research, & Evalua-
tion, 10, 1–9.
CROSS, W., MATTHIEU, M. M., LEZINE, D.,
& KNOX, K. L. (2010). Does a brief suicide preven-
tion gatekeeper training program enhance
observed skills? Crisis, 31, 149–159.
CUCCIARE, M. A., CURRAN, G. M., CRASKE,
M. G., ABRAHAM, T., MCCARTHUR, M. B., MARCH-
ANT-MIROS, K., ET AL. (2016). Assessing fidelity of
cognitive behavioral therapy in rural VA clinics:
Design of a randomized implementation effective-
ness (hybrid type III) trial. Implementation Science,
11, 65.
DAMSCHRODER, L. J., REARDON, C. M.,
AUYOUNG, M., MOIN, T., DATTA, S. K., SPARKS, J.
B., ET AL. (2017). Implementation findings from a
hybrid III implementation-effectiveness trial of the
Diabetes Prevention Program (DPP) in the Veter-
ans Health Administration (VHA). Implementation
Science, 12, 94.
DEANE, F. P., WILSON, C. J., & CIARROCHI,
J. (2001). Suicidal ideation and help-negation: Not
just hopelessness or prior help. Journal of Clinical
Psychology, 57, 901–914.
DEW, M. A., BROMET, E. J., BRENT, D., &
GREENHOUSE, J. B. (1987). A quantitative literature
review of the effectiveness of suicide prevention
centers. Journal of Consulting and Clinical Psychology,
55, 239–244.
FABRIGAR, L. R., WEGENER, D. T.,
MACCALLUM, R. C., & STRAHAN, E. J. (1999).
SAFETY PLANNING ON CRISIS LINES
Evaluating the use of exploratory factor analysis in
psychological research. Psychological Methods, 4,
272–299.
FLEISCHMANN, A., BERTOLOTE, J. M.,
WASSERMAN, D., DE LEO, D., BOLHARI, J., BOTEGA,
N. J., ET AL. (2008). Effectiveness of brief interven-
tion and contact for suicide attempters: A random-
ized controlled trial in five countries. Bulletin of the
World Health Organization, 86, 703–709.
GARSON, G. D. (Ed.) (2007). Factor analy-
sis. In Statnotes: Topics in multivariate analysis.
Retrieved April 13, 2009, from http://www2.chass.
ncsu.edu/garson/pa765/statnote.htm
GOULD, M. S., CROSS, W., PISANI, A. R.,
MUNFAKH, J. L., & KLEINMAN, M. (2013). Impact
of applied suicide intervention skills training on
the national suicide prevention lifeline. Suicide and
Life-Threatening Behavior, 43(6), 676–691.
GOULD, M. S., KALAFAT, J., HARRISMUN-
FAKH, J. L., & KLEINMAN, M. (2007). An evalua-
tion of crisis hotline outcomes part 2: Suicidal
callers. Suicide and Life-Threatening Behavior, 37,
338–352.
GOULD, M. S., LAKE, A. M., GALFALVY, H.,
KLEINMAN, M., MUNFAKH, J. L., WRIGHT, J., ET AL.
(2018). Follow-up with callers to the National Sui-
cide Prevention Lifeline: Evaluation of callers’ per-
ceptions of care. Suicide and Life-Threatening
Behavior, 48, 75–86.
GOULD, M. S., MUNFAKH, J. L. H., KLEIN-
MAN, M., & LAKE, A. M. (2012). National suicide
prevention lifeline: Enhancing mental health care
for suicidal individuals and other people in crisis.
Suicide and Life-Threatening Behavior, 42, 22–35.
HANGARTNER, R. B., TOTURA, C. M. W.,
LABOULIERE, C. D., GREGLEWICZ, K., & KARVER,
M. S. (2019). Benchmarking the “Question, Per-
suade, Refer” program against evaluations of estab-
lished suicide prevention gatekeeper trainings.
Suicide and Life-Threatening Behavior, 49(2), 353–
370.
HOLTON, E. F., BATES, R. A., & RUONA, W.
E. (2000). Development of a generalized learning
transfer system inventory. Human Resource Develop-
ment Quarterly, 11, 333–360.
HOLTON, E. F., CHEN, H. C., & NAQUIN, S.
S. (2003). An examination of learning transfer sys-
tems across organizational settings. Human
Resource Development Quarterly, 14, 459–482.
JOHNSON, R. M., FRANK, E. M., CIOCCA,
M., & BARBER, C. W. (2011). Training mental
healthcare providers to reduce at-risk patients’
access to lethal means of suicide: Evaluation of
the CALM Project. Archives of Suicide Research,
15, 259–264.
JOINER, T., KALAFAT, J., DRAPER, J., STOKES,
H., KNUDSON, M., BERMAN, A. L., ET AL. (2007).
Establishing standards for the assessment of sui-
cide risk among callers to the National Suicide
LABOULIERE ET AL.
Prevention Lifeline. Suicide and Life-Threatening
Behavior, 37, 353–365.
LABOULIERE, C. D., KLEINMAN, M., &
GOULD, M. S. (2015). When self-reliance is not
safe: Associations between reduced help-seeking
and subsequent mental health symptoms in suicidal
adolescents. International Journal of Environmental
Research and Public Health, 12, 3741–3755.
LABOULIERE, C. D., VASAN, P., KRAMER, A.,
BROWN, G., GREEN, K., KAMMER, J., ET AL. (2018).
“Zero Suicide” – A model for reducing suicide in
United States behavioral healthcare. Suicidologi, 23,
22–30.
LITMAN, R. E., FARBEROW, N. L., SHNEID-
MAN, E. S., HEILIG, S. M., & KRAMER, J. A. (1967).
Suicide-prevention telephone service. Journal of the
American Medical Association, 192, 21–25.
LUKE, D. (2004). Multilevel modeling. Thou-
sand Oaks, CA: Sage.
LUXTON, D. D., JUNE, J. D., & COMTOIS,
K. A. (2013). Can postdischarge follow-up con-
tacts prevent suicide and suicidal behavior? Crisis,
34, 32–41.
National Suicide Prevention Lifeline.
(2018). About. Retrieved January 1, 2018, from
https://suicidepreventionlifeline.org/about
RAUDENBUSH, S. W., & BRYK, A. S. (2002).
Hierarchical linear models: Applications and data anal-
ysis methods (2nd ed.). Thousand Oaks, CA: Sage.
STANLEY, B. (2017). Zero suicide: Implemen-
tation and evaluation in outpatient mental health clin-
ics. Paper presented at the IASR/AFSP Suicide
Research Summit, Las Vegas, Nevada.
STANLEY, B., & BROWN, G. K. (2012). Safety
planning intervention: A brief intervention to miti-
gate suicide risk. Cognitive and Behavioral Practice,
19, 256–264.
13
STANLEY, B., BROWN, G. K., CURRIER, G.
W., LYONS, C., CHESIN, M., & KNOX, K. L. (2015).
Brief intervention and follow-up for suicidal
patients with repeat emergency department visits
enhances treatment engagement. American Journal
of Public Health, 105, 1570–1572.
STANLEY, B., CHAUDHURY, S. R., CHESIN,
M., PONTOSKI, K., BUSH, A. M., KNOX, K. L.,
ET AL. (2016). An emergency department interven-
tion and follow-up to reduce suicide risk in the VA:
Acceptability and effectiveness. Psychiatric Services,
67, 680–683.
Suicide Prevention Resource Center.
(2018). Evidence-based prevention website.
Accessed on January 1, 2018 from https://www.
sprc.org/keys-success/evidence-based-prevention.
TABACHNICK, B. G., & FIDELL, L. S. (2001).
Principal components and factor analysis. In B. G.
Tabachnick, & L. S. Fidell (Eds.), Using multivari-
ate statistics (pp. 582–652). Boston, MA: Allyn &
Bacon.
WYMAN, P. A., BROWN, C. H., INMAN, J.,
CROSS, W., SCHMEELK-CONE, K., GUO, J., ET AL.
(2008). Randomized trial of a gatekeeper program
for suicide prevention: 1-year impact on secondary
school staff. Journal of Consulting and Clinical Psy-
chology, 76, 104–115.
ZATZICK, D. F., RUSSO, J., DARNELL, D.,
CHAMBERS, D. A., PALINKAS, L., VAN EATON, E.,
ET AL. (2015). An effectiveness-implementation
hybrid trial study protocol targeting posttraumatic
stress disorder and comorbidity. Implementation
Science, 11, 58.
Manuscript Received: November 14, 2018
Revision Accepted: March 20, 2019