Received: 17 July 2017 Revised: 27 December 2017 Accepted: 20 January 2018

DOI: 10.1002/da.22730

RESEARCH ARTICLE

Digital phenotyping of suicidal thoughts

Evan M. Kleiman PhD1 Brianna J. Turner PhD2 Szymon Fedor PhD3
Eleanor E. Beale BA4,5 Rosalind W. Picard ScD3 Jeff C. Huffman MD4,5
Matthew K. Nock PhD1,4,6

1 Department of Psychology, Harvard University,

Background: To examine whether there are subtypes of suicidal thinking using real-time digital
Cambridge, MA, USA
2 Department of Psychology, University of monitoring, which allows for the measurement of such thoughts with greater temporal granularity
Victoria, Victoria, Canada than ever before possible.
3 Media Lab, Massachusetts Institute of
Methods: We used smartphone-based real-time monitoring to assess suicidal thoughts four times
Technology, Cambridge, MA, USA
4 Department of Psychiatry, Massachusetts
per day in two samples: Adults who attempted suicide in the past year recruited from online

General Hospital, Boston, MA, USA
5 Department of Psychiatry, Harvard Medical
School, Cambridge, MA, USA
6 Cambridge Computational Clinical Psychology
Organization (C3PO), Cambridge, MA, USA
Correspondence
Evan M. Kleiman, Department of Psychology,
Harvard University, Cambridge, MA 02138.
Email: ekleiman@fas.harvard.edu
Funding information
Grant sponsor: Chet and Will Griswold Suicide
Prevention Fund; Grant sponsor: For the Love of
Travis; Grant sponsor: Pershing Square Venture
Fund for Research on the Foundations of Human
Behavior; Grant sponsor: John D. and Catherine
T. MacArthur Foundation.
forums (n = 51 participants with a total of 2,889 responses, surveyed over 28 days; ages ranged
from 18 to 38 years) and psychiatric inpatients with recent suicidal ideation or attempts (n = 32
participants with a total of 640 responses, surveyed over the duration of inpatient treatment
[mean stay = 8.79 days], ages ranged 23–68 years). Latent profile analyses were used to identify
distinct phenotypes of suicidal thinking based on the frequency, intensity, and variability of such
thoughts.
Results: Across both samples, five distinct phenotypes of suicidal thinking emerged that differed
primarily on the intensity and variability of suicidal thoughts. Participants whose profile was char-
acterized by more severe, persistent suicidal thoughts (i.e., higher mean and lower variability
around the mean) were most likely to have made a recent suicide attempt.
Conclusions: Suicidal thinking has historically been studied as a homogeneous construct, but using
newly available monitoring technology we discovered five profiles of suicidal thinking. Key ques-
tions for future research include how these phenotypes prospectively relate to future suicidal
behaviors, and whether they represent remain stable or trait-like over longer periods.

KEYWORDS
assessment/diagnosis, computer/Internet technology, depression, suicide/self harm, web based

1 INTRODUCTION
The question of why people behave in ways that are harmful to them-
selves has puzzled scholars for thousands of years. The decision of
whether to live or die has been called the “fundamental question of
philosophy” (Camus, 1942) and has been the focus of scholarly work
by most major philosophers throughout history (e.g., Kant, Sartre,
Locke, Hume). In the sciences, the existence of suicidal thoughts and
behaviors has presented a fundamental challenge to the belief that
human and animal behavior is motivated by an innate and ever-present
drive for self-preservation and gene survival (Dawkins, 1976; Lorenz,
1963; Wilson, 1978). Despite centuries of scholarly consideration and
scientific investigation, key questions about suicide remain surpris-
ingly unanswered, and it continues to be one of the leading causes of
death worldwide. Indeed, approximately 9% of adults around the world
report that they have seriously considered suicide at some point in
their lives (Nock et al., 2008), and suicide accounts for over 800,000
deaths each year (World Health Organization, 2016a), more than all
wars and other forms of interpersonal violence combined—meaning
that we each are more likely to die by our own hand than by some-
one else's (World Health Organization, 2016b). More alarming is that
suicide is projected to become an even greater contributor to the
global burden of disease in the coming decades (Mathers & Loncar,
2006). Understanding this perplexing aspect of human nature is one of
the greatest challenges facing our society. Some of the key questions
regarding suicide that remain unanswered involve understanding and
classifying the everyday experience of individuals at risk for suicide.
In most areas of science (biology, chemistry, ethology, etc.), we
obtain an understanding of phenomena of interest by directly observ-
ing and studying them (Kagan, 1967; Lorenz, 1981; Tinbergen, 1951,
1974). Historically, such an approach has not been possible in the study
of suicidal thoughts and behaviors because they occur privately and

Depress Anxiety. 2018;1–8. wileyonlinelibrary.com/journal/da 

c 2018 Wiley Periodicals, Inc. 1
2 KLEIMAN ET AL .
episodically in a person's life, outside of the scope of standard psy-
chological assessment methods. Moreover, the tools used to assess
suicidal thoughts have been until now limited to intermittent assess-
ments separated by weeks, months, or years, which does not address
the fact that suicidal thoughts can be highly variable over a few hours
(Bagge, Littlefield, Conner, Schumacher, & Lee, 2014; Kleiman et al.,
2017) and that suicide attempts can occur in response to rapidly esca-
lating thoughts over time periods as short as a day (Millner, Lee, &
Nock, 2016). However, recent advances in smartphone-based real-
time monitoring technology (i.e., ecological momentary assessment
[EMA]) (Shiffman, Stone, & Hufford, 2008) have made it possible for
the first time to overcome these limitations by allowing individuals to
report on suicidal thoughts as they naturally occur in a variety of set-
tings (e.g., over the course of clinical care, in a person's day-to-day life),
which has been done in a small handful of studies to date (for review,
see Kleiman & Nock, 2018). Indeed, the use of smartphones and related
portable devices are providing new opportunities for “digital pheno-
typing,” that is, providing real-time characterization and quantification
of human behavior in situ (Bidargaddi et al., 2017; Onnela & Rauch,
2016; Torous, Onnela, & Keshavan, 2017).
In two recent studies using real-time monitoring to observe suici-
dal thoughts, our group (Kleiman et al., 2017) and others (Hallensleben
et al., 2017) found that thoughts of suicide vary considerably over short
periods of time (e.g., hour to hour). Here we examine whether there are
distinct profiles or subtypes of suicidal thinking by examining hour-to-
hour changes in the reports of suicidal thoughts over the span of sev-
eral weeks. Prior studies aimed at identifying subtypes of people at risk
for suicide have done so using different profiles of risk factors (Bagge,
Littlefield, & Glenn, 2017; Ginley & Bagge, 2017; Hamza & Willoughby,
2013; Herres, Kodish, Fein, & Diamond, 2017; Logan, Hall, & Karch,
2011).
Although several studies find different phenotypes based on risk
factors for suicidal thoughts and several others have examined profiles
of suicidal thinking based on the trajectory of suicidal thinking mea-
sured weeks or months apart (Czyz & King, 2015; Goldston et al., 2016;
Wolff et al., 2018), no prior studies have examined whether there are
different phenotypes of suicidal thoughts based on the actual experi-
ence of thoughts itself. Recent theoretical work by Bernanke, Stanley,
and Oquendo (2017) suggests that there may be at least two subtypes
of the experience of suicidal thinking. One proposed subtype is charac-
terized by large fluctuations in severity of suicidal thoughts in response
to life stress. The other proposed subtype is characterized by persis-
tent levels of suicidal thinking that do not fluctuate in response to life
events. If such subtypes or classes of thoughts are observed, they may
provide useful information for testing whether different phenotypes
have different predictors, courses, and responsiveness to treatments,
providing traction in areas of suicide research that have been relatively
stagnant. Accordingly, the goal of this study was to see if we could clas-
sify phenotypes of suicidal thinking based upon the experience of sui-
cidal thinking itself. To do this, we used data from two samples of peo-
ple at high risk for suicide (recent attempters from the community and
suicidal inpatients), who completed digital smartphone monitoring for
28 days (community members) or the length of inpatient treatment
(inpatients).
2 METHOD
2.1 Participants and recruitment
2.1.1 Sample 1
The first sample consisted of 51 adults who had attempted suicide
in the past year (79% female, mean age = 23.59 years, SD = 4.74
years, range 18–38 years). Regarding race, 72.54% of the sample self-
identified as being of European decent, 9.80% Hispanic, 7.84% Asian,
and the rest identified as another race. The majority of the sample
(96.07%) had at least a high-school degree. Participants were recruited
from forums relating to self-harm or suicide on the website Reddit
(www.reddit.com). Inclusion criteria included a suicide attempt with at
least some intent to die occurring in the past year, fluency in English,
age 18+ years, and regular access to an Internet-capable smartphone.
To determine eligibility, participants were first asked to complete a
brief screener linked to the study ad. Eight hundred fifty-four peo-
ple completed the screener, 103 of whom qualified for the study, 90
of whom were interested in the study (i.e., 87% of those who quali-
fied). Of those who were interested and qualified, 54 people began the
study (60% of those who were interested and qualified), 51 (56% of
those who were interested and qualified) of whom completed at least
three or more consecutive data points. Consecutive data points were
required to calculate the variability statistics used in our analyses.
2.1.2 Sample 2
The second sample consisted of 32 adult inpatients who were hospi-
talized at the psychiatric inpatient unit at Massachusetts General Hos-
pital for a recent suicide attempt or severe suicidal thoughts (43.3%
female, mean age = 42.53 years, SD = 12.88 years, range 23–68 years).
Regarding race, the sample was 81.25% European decent, 6.25% His-
panic, 6.25% Asian, and the rest identified as another race. Inclusion
criteria were admission due to a suicide attempt or severe suicidal
thoughts and fluency in English (we loaned smartphones to those who
did not have or own one). A study staff member approached any new
admissions from the previous day that met inclusion criteria. Fifty-
three patients were approached during the study period, 41 (77.3%)
of whom were willing to participate and were consented for the study.
Of those consented, 32 were included in the study (78% of those con-
sented). Reasons for not being were as follows: one potential partici-
pant was discharged unexpectedly early, two did not appear to under-
stand the study, two did not answer any of the smartphone surveys, and
four did not complete three or more consecutive responses.
2.2 Procedure
In both studies, participants completed a brief set of baseline question-
naires followed by a period of smartphone-based real-time monitor-
ing where they were prompted to report on any experiences of suici-
dal thoughts four times per day each day (other experiences that were
reported, such as mood states, are not relevant to the present study).
Prompts were sent randomly within predefined intervals (i.e., times
when the participant was awake). Below, we discuss areas of difference
between the two studies.
KLEIMAN ET AL .
2.2.1 Sample 1
Participants in the first sample completed real-time monitoring for a
28-day period. Prompts were delivered and responses collected via
Mobile EMA software (www.ilumivu.com), which is compatible with
both Android and iPhone smartphones. Participants were compen-
sated with a $40 (USD) gift card to Amazon.com, with a $10 bonus for
completing more than 75% of the prompts.
2.2.2 Sample 2
Participants in the second sample completed real-time monitoring
for the duration of their inpatient stay (mean stay = 8.79 days,
SD = 8.23 days, range = 2–46 days, median = 7 days). Prompts
were delivered and responses collected via MovisensXS EMA software
(www.movisens.com). MovisensXS is only compatible with Android
smartphones, so we lent compatible phones to participants for the
duration of the study if they did not have access to a smartphone, or
if they owned an incompatible phone. Participants were compensated
with $10 (paid in cash) for each day they were in the study.
2.3 Measures
2.3.1 Baseline
Participants completed a brief demographics screener and the self-
report version of the Self-Injurious Thoughts and Behaviors Interview
(SITBI; Nock, Holmberg, Photos, & Michel, 2007). The SITBI was used
to assess suicide history (i.e., age of first attempt, number of lifetime
attempts, date of most recent suicide attempt) and to also confirm our
main inclusion criterion (past year suicide attempt).
2.3.2 Real-time monitoring
At each prompt, participants were presented with three items assess-
ing: (1) the desire to die by suicide (How intense is your desire to kill
yourself right now?), (2) the intention to die by suicide (How strong is
your intention to kill yourself by suicide right now?), and (3) the abil-
ity to resist the urge to die by suicide (“How strong is your ability to
resist the urge to kill yourself right now?,” which was reverse coded).
Each item for the first sample was on a 0 (not strong [intense] at all) to
4 (very strong [intense]) scale. Each item for the second sample was on
a 0 (not strong [intense] at all) to 9 (very strong [intense]) scale. Since
the items were on different scales, we could not combine samples. The
three items were summed such that higher scores equaled more severe
suicidal thoughts. These three component items were strongly inter-
correlated in both samples (|r| range from 0.54 to 0.89, all P < .001).
2.4 Analytic strategy
We used latent profile analysis (LPA) to test for the existence of distinct
profiles of suicidal thinking in the real-time monitoring data described
above. LPA is useful to classify groups of individuals based upon a
set of continuous criterion variables, creating phenotypes of suicidal
thoughts. There are several guidelines to assess the correct number of
profiles: The Bayesian information criterion (BIC), a measure of model
fit; entropy, a measure of precision of classification of individuals into
3
profiles; and the bootstrapped k − 1 likelihood ratio test (LRT), a mea-
sure that compares a model with the current number (k) of profiles to
one with one fewer (k − 1) profile to determine if precision is improved
by the addition of an extra profile. The optimal solution is indicated
by the model with the lowest BIC, largest entropy, and a significant k
− 1 test. We used as indicators several measurements that quantify
the patterns of suicidal thoughts experienced by each individual: (1)
Mean scores across the three suicidal thoughts items, indicating aver-
age severity of suicidal thoughts across the study, (2) within-person
standard deviations of each person's suicidal thought scores, indicat-
ing average within-person dispersion of suicidal thoughts around the
mean, (3) the maximum total score for suicidal thought items, indi-
cating the greatest intensity of suicidal thoughts experienced during
the monitoring period, (4) the percent of prompts for which a nonzero
score on suicidal thoughts was reported, indicating relative frequency
of suicidal thinking, and (5) the root mean square of successive dif-
ferences (RMSSD; von Neumann, Kent, Bellinson, & Hart, 1941), indi-
cating the average stability (or magnitude of change) of scores from
one measurement occasion to the next. The RMSSD is ideal to quan-
tify nonlinear variability in repeated measures and is interpreted such
that larger values equal more variability from one point to the next
and, if graphed, would correspond to a more jagged pattern. After we
conducted the LPA, we examined if the phenotypes differed on suicide
history variables using ANOVAs (analysis of variance) or chi-square
tests. The LPA was performed in Mplus version 7 (Muthén, & Muthén,
2014). All other analyses were performed and figures were created in R
(R Core Team, 2016).
3 RESULTS
Participants in Sample 1 completed a total of 2,889 responses
(M = 56.59 responses per participant, SD = 37.77) across a total of
1,311 days for an average of 2.20 responses per participant, per day
(out of four possible prompts; 55.0% response rate). Participants in
Sample 2 completed a total of 640 responses (M = 20 responses per
participant, SD = 17.48) across a total of 281 days for an average of
2.28 responses per participant, per day (out of four possible prompts;
57.0% response rate).
3.1 Subtypes of suicidal thinking (Sample 1)
Table 1 contains fit statistics for all profiles. The LPA converged on a
five-profile model (BIC = 1,153.27, entropy = 0.961, k − 1 LRT = 58.95,
P < .001). Although the six-profile model had a lower BIC and higher
entropy than the five-profile model, as well as a significant k − 1 LRT
value, the five-profile model was still preferable for a few reasons.
First, the six-profile model caused Mplus to produce nonidentifica-
tion warnings that likely indicated a model with too many profiles.
Second, the six-profile model produced very small cell sizes (e.g., one
phenotype profile had only two participants). Third, although there
was an improvement in model fit from the five-profile to six-profile
model, the increment in model fit was relatively smaller than the incre-
ment in fit seen in other models (e.g., entropy increased by 0.012 from
4 KLEIMAN ET AL .

TA B L E 1 Latent profile analyses fit statistics and phenotype profile membership

Sample 1 (Past-year attempters)

Fit statistics Phenotype profile membership
Profiles BIC Entropy k − 1 LRT 1 2 3 4 5 6
2 1,216.14 0.931 131.64, P < .001 51.92% 46.15%
3 1,186.23 0.911 53.62, P < .001 40.38% 32.69% 25.00%
4 1,172.32 0.921 37.59, P < .001 28.85% 28.85% 23.08% 17.31%
5 1,153.27 0.961 58.95, P < .001 28.85% 26.92% 17.31% 17.31% 7.69%
6 1,148.73 0.973 28.45 P < .001 28.85% 26.92% 19.23% 15.38% 5.77% 3.85%
Sample 2 (Suicidal inpatients)
Fit statistics Phenotype profile membership
Profiles BIC Entropy k − 1 LRT 1 2 3 4 5 6
2 939.56 0.916 69.31, P < .001 50.00% 50.00%
3 898.54 0.911 61.82, P < .001 43.75% 40.63% 15.63%
4 888.01 0.993 31.32, P < .001 43.75% 28.13% 15.63% 12.50%
5 874.43 0.991 34.37, P < .001 28.13% 28.13% 18.75% 12.50% 12.50%
6 872.52 0.993 22.28 P = .069 28.13% 25.00% 15.63% 12.50% 12.50% 6.25%

Notes: BIC = Bayesian Information Criterion; LRT = likelihood ratio test. Profile numbers sorted by largest to smallest, but do not necessarily correspond to
the same profile from model to model (e.g., profile 3 in the three-profile model is not the same exact profile as profile 3 in the four-profile model).

2385 2406 2428 2444 2841 2842 2844 2885

12
8
4
0
2159 2259 2436 2438 2439 2440 2441 2443
12
8
Momentary rating of suicidal thinking

4
0
2460 2851 2852 2882 2883 2157 2158 2257
12
8
4
0
2260 2369 2410 2416 2417 2430 2437 2846
12
8
4
0
2848 2888 2890 2379 2383 2431 2435 2371
12
8
4
0
2414 2415 2418 2429 2432 2458 2847 2889
12
8
4
0
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
Observation Number (approx 4−8 hours apart)

1: Low mean, low variability 2: Low mean, moderate variability 3: Moderate mean, high variability
Phenotype Profile:
4: High mean, low variability 5: High mean, high variability

F I G U R E 1 Individual time series plots of suicidal thoughts in Sample 1 (raw scores).

Note: horizontal line = mean.

the five-profile to six-profile model, but increased by 0.040 from the
four-profile model to the five-profile model). Thus, on balance, the five-
profile model was preferable. Examination of the data from individ-
ual participants classified by these five profiles (Fig. 1, Table 2) reveals
that they are distinguished by their differences in mean and varia-
tion around that mean: (1) low mean, low variability; (2) low mean,
moderate variability; (3) moderate mean, high variability; (4) high
mean, low variability; and (5) high mean, high variability. Statistical
comparisons of these five groups revealed no differences among these
five phenotypes in terms of age of first suicide attempt and number
of lifetime suicide attempts (Table 1). However, the phenotype typi-
fied by high mean and low variability (Phenotype 4) has a substan-
tially higher proportion of individuals who had attempted suicide in
the month before the study (and contained the one participant who
attempted suicide in the week before the study) than any other phe-
notype. The phenotype typified by high mean and high variability
KLEIMAN ET AL . 5

TA B L E 2 Comparison of phenotype profiles

Sample 1 (Past-year attempters)

1 (n = 9) 2 (n = 15) 3 (n = 14) 4 (n = 4) 5 (n = 9)
Low mean,low Low mean, mod. Mod. mean, high High mean, High mean, high
Profile variability variability variability low variability variability Test stat. P
Thoughts (LPA indicators)
RMSSD 0.84a 2.01b 3.08c 1.64a,b 3.38c 45.38 < .001
Mean 0.42a 1.27a 2.34b 6.00c 5.51c 69.42 < .001
SD 0.68a 1.66b 2.78c 1.50b 2.83c 59.52 < .001
Max 2.70a 6.27b 9.93c 9.50c 11.22c 57.82 < .001
%>0 28.95a 47.03a,b 58.56b 99.94c 94.96c 21.13 < .001
Suicide history
Age of first attempt 17.89 17.67 14.08 16.50 13.57 2.50 .057
No. of lifetime attempts 3.00 4.07 5.08 3.25 6.71 1.44 .237
Percentage of past month 20%a 6.7%a 28.6%a 75%a 0%b 11.57 .021
attempt
Percentage of past week 0%a 0%a 0%a 25%b 0%a 11.23 .024
attempt
Sample 2 (suicidal inpatients)
1 (n = 4) 2 (n = 9) 3 (n = 6) 4 (n = 9) 5 (n = 4)
Low mean, Mod. mean, low Mod. mean, mod. High mean, mod. High mean, high
Profile low variability variability variability variability variability Test stat. P
Thoughts (LPA indicators)
RMSSD 0.69a 1.49a 4.09b 3.63b 6.81c 63.56 < .001
Mean 0.54a 3.73b 3.07a,b 8.47c 12.00d 19.92 < .001
SD 0.83a 1.22b 3.40c 3.29d 5.12e 39.78 < .001
Max 2.50a 6.33a 9.83b 15.00b 23.00c 29.40 < .001
%>0 21.99a 99.5b 62.73c 99.11b 99.57b 39.99 < .001
Suicide history
Age of first attempt 21.50 43.33 29.20 24.33 23.00 2.20 .399
No. of lifetime attempts 3.25 2.17 2.40 6.67 8.33 1.07 .108
Percentage of past month 50.00% 66.70% 83.33% 66.70% 100.00% 2.38 .666
attempt
Percentage of past week 25.0% 50.0% 40.0% 66.7% 66.7% 2.20 .699
attempt
Notes: Numbers in rows that do not share subscripts are significantly different at P < .05. F[4,47], F[4,27] for Studies 1 and 2 ANVOAs, respectively. df = 4 for
𝜒 2 tests in both studies.

(Phenotype 5) contained zero members who attempted suicide in the
month before the study. Taken together, this indicates that among
those who had higher mean levels of suicidal thoughts, a lower degree
of variation around that mean was indicative of someone who had
more recently attempted suicide.
higher rate of recent suicide attempt in Phenotype 4 was not signif-
icantly higher than all other groups in this sample. Figure 2 shows a
visual depiction of the profiles, which looked similar to those from
Sample 1.

3.2 Results from replication sample (Sample 2)
The LPA again converged on a five-profile model (BIC = 874.43,
entropy = 0.991, k − 1 LRT = 34.37, P < .001). Table 1 contains fit
statistics for all profiles. Examination of the data from individual par-
ticipants classified by these five profiles revealed the same pattern
of findings observed in Sample 1 in terms of differences in mean and
variability of suicidal thinking and in group differences in history of
suicide attempts (Table 2), with the only difference being that the
4 DISCUSSION
Suicide is among the leading causes of death worldwide and nearly
10% of people report thinking about suicide at some point in their
lives. Unfortunately, our understanding of suicide—and ability to pre-
dict and prevent it—has been hindered by a lack of information about
the basic nature of suicidal thoughts. Using smartphone-based dig-
ital phenotyping methods, we identified five distinct phenotypes of
suicidal thinking—a pattern that replicated across two samples with
6 KLEIMAN ET AL .

3 12 20 21 2 22 24 33

20
10
Momentary rating of suicidal thinking

0
40 42 44 46 47 4 5 25

20
10
0
26 28 31 13 19 27 29 32

20
10
0
34 35 39 41 6 16 17 30

20
10
0
5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20 5 10 15 20
Observation Number (approx 4−8 hours apart)

1: Low mean, low variability 2: Moderate mean, low variability 3: Moderate mean, moderate variability
Phenotype Profile:
4: High mean, moderate variability 5: High mean, high variability

F I G U R E 2 Individual time series plots of suicidal thoughts in Sample 2 (raw scores).

Note: horizontal line = mean.

different levels of acuity (i.e., community-dwelling adults vs. psychiatri-
cally hospitalized inpatients). The phenotypes differed primarily in the
average severity of suicidal thoughts across measurement occasions,
and the magnitude of within-person variability around that average.
Moreover, the phenotype typified by more severe (i.e., higher mean)
and more stable (i.e., lower RMSSD) suicidal ideation contained the
highest proportion of individuals who had attempted suicide in the
past month in the community sample. This finding was not statisti-
cally significant in the second sample, although the higher acuity of
the sample may have resulted in a ceiling effect to detect group dif-
ferences in recency of suicide attempts. Recent real-time monitor-
ing studies have shown that there is substantial heterogeneity in the
experience of suicidal thoughts (Kleiman et al., 2017), and the cur-
rent study extends this earlier work by showing that there is order
to this heterogeneity. Overall, these findings have important impli-
cations for future research aimed at better understanding the phe-
nomenology of suicidal thinking and for clinical treatment of suicidalx
individuals.
This work is generally in line with Bernanke, Stanley, and Oquendo's
(2017) proposal that there are two distinct phenotypes of suicidal
thinking, typified by high versus low levels of variability. Although we
found five phenotypes, one of the clearest descriptors of the phe-
notypes was how variable suicidal thinking was. Thus, it might be
that the inclusion of other factors like mean level of suicidal thinking
adds nuance to Bernake et al.’s theorized phenotypes (and indeed, the
authors noted that more than two phenotypes might be a possibility).
Relatedly, it may be that there are two superordinate phenotypes (high
vs. low variability) and the other phenotypes we found describe sub-
types of these superordinate phenotypes. One way in which our study
could not fully test Bernake et al.’s model is that we did not have a mea-
sure of stress that allowed us to see if the peaks in suicidal thinking
among those in the high variability phenotypes were predicted by
stress. Additionally, our findings are also in line with a line of studies
showing that more stable suicidal thinking is associated with increased
risk for suicidal behavior. For example, one study found that among mil-
itary servicemembers who reported having thoughts of suicide, those
who had episodes of suicidal thinking lasting 5 hrs or longer were at
more than double the risk of those with shorter duration episodes
of suicidal thinking to act on their suicidal thoughts (Nock et al.,
in press).
These findings should be viewed in the context of several limita-
tions. First, although the number of participants in each of the five
phenotypes was evenly distributed in both samples, the samples were
relatively small leading to somewhat small absolute numbers of partici-
pants per class. The replication of findings across two different samples
increases confidence in the findings, but additional confirmatory stud-
ies are needed. Second, we followed participants in each sample for no
more than 1 month at a time and so in this study we were unable to
assess if one phenotype versus another predicts suicidal behaviors or
other outcomes in the future. We would expect the more severe phe-
notypes (i.e., those with more severe/intense suicidal thoughts) to be at
greater risk for another suicide attempt; however, future studies using
longer-term follow-up periods are needed to test this. Third, these pro-
files may be influenced by the characteristics of the samples in which
they were collected. For example, the inpatient in Sample 2 may have
been less inclined to disclose severe suicidal thinking out of concerns
of further clinical intervention or that they would not be released
from the hospital. Thus, further replication in larger and more diverse
samples is needed. Fourth, although there were many similarities
between the two samples, there may have been important differences
in the how participants interacted with the two software packages. For
example, due to differences in how items were displayed on the screen,
KLEIMAN ET AL .
the software used in Sample 1 used a five-point scale for items,
whereas the software used in Sample 2 used a 10-point scale. Given
that some work finds that data from scales with more points tend to be
more normally distributed (Leung, 2011), it may be that the 10-point
scale used in Sample 2 was better equipped to detect variability in sui-
cidal thinking. Finally, a broader issue that extends beyond the scope of
this manuscript is the choice of terminology used to address the sub-
types of suicidal thinking. Here, we used “digital phenotype” as it most
closely matches the work on which this manuscript is based (Onnela, &
Rauch, 2016; Torous et al., 2017), although other terminology may also
apply here (e.g., “digital footprint”; Bidargaddi et al., 2017).
Although this study represents a first step toward developing a
digital phenotype, there are several important directions for future
studies. First, although we were primarily interested in quantifying
variability and seeing if meaningful phenotypes could be established
from this quantification, there may be additional ways to look at these
data. For example, growth mixture modeling would allow future stud-
ies to assess whether individuals have different trajectories of change
in suicidal ideation (e.g., over the inpatient or postdischarge period).
Second, future studies should explore the full range of available data
(e.g., GPS data, call logs) that may further refine phenotypes of suicidal
ideation (Kleiman & Nock, 2017). Third, in both samples we conceptu-
alized suicidal thinking with a three-item composite that assessed one's
desire to die by suicide, intent to die by suicide, and ability to resist
the urge to die by suicide. However, additional conceptualizations of
suicidal thinking have been linked to future suicidal behavior (e.g., per-
sistence of suicidal thoughts; controlability of suicidal thoughts; Nock
et al., in press). Accordingly, future studies should examine whether
other conceptualizations of suicidal thinking help to further distinguish
phenotypes of suicidal thinking. Finally, studies with longer follow-up
lengths may help assess stability or change in phenotypes. If studies
with longer follow-up periods find that some individuals’ phenotype
membership changes over time, it may be useful to examine factors
that predict these changes.
Clinically, these findings suggest that it is important to not just
assess clients’ average level of suicidal thinking but also variability
around that average. This is important because those who have had
severe and stable suicidal thinking (as in the high mean/low variabil-
ity phenotype in this study) may be at greater risk of engaging in sui-
cidal behavior, although future studies are needed to fully test this
idea. In conclusion, the results from this initial study suggest that
real-time monitoring of suicidal individuals is feasible and can pro-
vide valuable new information about suicidal thoughts—supporting the
conduct of future, longer-term digital phenotyping studies of suicidal
behavior.
ACKNOWLEDGMENTS
The research was supported by the Chet and Will Griswold Suicide
Prevention Fund (MKN), For the Love of Travis (MKN), the Persh-
ing Square Venture Fund for Research on the Foundations of Human
Behavior (EMK), and the John D. and Catherine T. MacArthur Founda-
tion (MKN).
7
CONFLICT OF INTEREST
The authors have no conflicts of interest to report.
ORCID
Evan M. Kleiman PhD http://orcid.org/0000-0001-8002-1167
REFERENCES
Bagge, C. L., Littlefield, A. K., Conner, K. R., Schumacher, J. A., &
Lee, H.-J. (2014). Near-term predictors of the intensity of suici-
dal ideation: An examination of the 24 h prior to a recent suicide
attempt. Journal of Affective Disorders, 165, 53–58. https://doi.org/10.
1016/j.jad.2014.04.010
Bagge, C. L., Littlefield, A. K., & Glenn, C. R. (2017). Trajectories of affective
response as warning signs for suicide attempts: An examination of the
48 hours prior to a recent suicide attempt. Clinical Psychological Science,
5(2), 259–271. https://doi.org/10.1177/2167702616681628
Bernanke, J. A., Stanley, B. H., & Oquendo, M. A. (2017). Toward fine-grained
phenotyping of suicidal behavior: The role of suicidal subtypes. Molec-
ular Psychiatry, 22(8), 1080–1081. https://doi.org/10.1038/mp.2017.
123
Bidargaddi, N., Musiat, P., Makinen, V.-P., Ermes, M., Schrader, G., &
Licinio, J. (2017). Digital footprints: Facilitating large-scale environ-
mental psychiatric research in naturalistic settings through data
from everyday technologies. Molecular Psychiatry, 22(2), 164–169.
https://doi.org/10.1038/mp.2016.224
Camus, A. (1942). The myth of Sisyphus and other essays (J. O'Brien, Trans.).
New York: Vintage.
Czyz, E. K., & King, C. A. (2015). Longitudinal trajectories of suicidal
ideation and subsequent suicide attempts among adolescent inpa-
tients. Journal of Clinical Child and Adolescent Psychology, 44(1), 181–193.
https://doi.org/10.1080/15374416.2013.836454
Dawkins, R. (1976). The selfish gene. New York, NY: Oxford University Press.
Ginley, M. K., & Bagge, C. L. (2017). Psychiatric heterogeneity of recent sui-
cide attempters: A latent class analysis. Psychiatry Research, 251, 1–7.
https://doi.org/10.1016/j.psychres.2017.02.004
Goldston, D. B., Erkanli, A., Daniel, S. S., Heilbron, N., Weller, B. E., &
Doyle, O. (2016). Developmental trajectories of suicidal thoughts and
behaviors from adolescence through adulthood. Journal of the Amer-
ican Academy of Child and Adolescent Psychiatry, 55(5), 400–407.e1.
https://doi.org/10.1016/j.jaac.2016.02.010
Hallensleben, N., Spangenberg, L., Forkmann, T., Rath, D., Hegerl, U.,
Kersting, A., … Glaesmer, H. (2017). Investigating the dynamics of
suicidal ideation: Preliminary findings from a study using ecolog-
ical momentary assessments in psychiatric inpatients. Crisis, 1–5.
https://doi.org/10.1027/0227-5910/a000464
Hamza, C. A., & Willoughby, T. (2013). Nonsuicidal self-injury and suicidal
behavior: A latent class analysis among young adults. PLoS One, 8(3),
e59955. https://doi.org/10.1371/journal.pone.0059955
Herres, J., Kodish, T., Fein, J., & Diamond, G. (2017). Screening to identify
groups of pediatric emergency department patients using latent class
analysis of reported suicidal ideation and behavior and non-suicidal
self-injury. Archives of Suicide Research, 1–12. (Epub ahead of print).
https://doi.org/10.1080/13811118.2017.1283264
Kagan, J. (1967). On the need for relativism. American Psychologist, 22(2),
131–142. https://doi.org/10.1037/h0024311
Kleiman, E. M., & Nock, M. K. (2017). Advances in scientific possibilities
offered by real-time monitoring technology. Psychiatry, 80(2), 118–124.
https://doi.org/10.1080/00332747.2017.1325661
8 KLEIMAN ET AL .
Kleiman, E. M., & Nock, M. K. (2018). Real-time assessment of suici-
dal thoughts and behaviors. Current Opinion in Psychology, 22, 33–37.
https://doi.org/10.1016/j.copsyc.2017.07.026
Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Huffman, J. C., &
Nock, M. K. (2017). Examination of real-time fluctuations in suicidal
ideation and its risk factors: Results from two ecological momentary
assessment studies. Journal of Abnormal Psychology, 126(6), 726–738.
https://doi.org/10.1037/abn0000273
Leung, S.-O. (2011). A comparison of psychometric properties and nor-
mality in 4-, 5-, 6-, and 11-point Likert scales. Journal of Social
Service Research, 37(4), 412–421. https://doi.org/10.1080/01488376.
2011.580697
Logan, J., Hall, J., & Karch, D. (2011). Suicide categories by patterns of known
risk factors: A latent class analysis. Archives of General Psychiatry, 68(9),
935–941. https://doi.org/10.1001/archgenpsychiatry.2011.85
Lorenz, K. Z. (1963). On aggression. New York: Harcourt Brace.
Lorenz, K. Z. (1981). The foundations of ethology. Vienna: Springer.
Mathers, C. D., & Loncar, D. (2006). Projections of global mortality and
burden of disease from 2002 to 2030. PLoS Medicine, 3(11), e442.
https://doi.org/10.1371/journal.pmed.0030442
Millner, A. J., Lee, M. D., & Nock, M. K. (2016). Describing and mea-
suring the pathway to suicide attempts: A preliminary study. Suicide
and Life-Threatening Behavior, 47(3), 353–369. https://doi.org/10.1111/
sltb.12284
Muthén, L. K., & Muthén, B. O. (2014). Mplus user's guide (7th ed.). Los Ange-
les, CA: Muthén & Muthén.
Nock, M. K., Borges, G., Bromet, E. J., Alonso, J., Angermeyer, M., Beautrais,
A., … Williams, D. R. (2008). Cross-national prevalence and risk factors
for suicidal ideation, plans, and attempts. British Journal of Psychiatry,
192, 98–105. https://doi.org/10.1192/bjp.bp.107.040113
Nock, M. K., Holmberg, E. B., Photos, V. I., & Michel, B. D. (2007). Self-
injurious thoughts and behaviors interview: Development, reliability,
and validity in an adolescent sample. Psychological Assessment, 19(3),
309–317. https://doi.org/10.1037/1040-3590.19.3.309
Nock, M. K., Millner, A. J., Joiner, T. E., Gutierrez, P. M., Han, G., Hwang, I.,
… Kessler, R. C. (in press). Risk factors for the transition from suicide
ideation to suicide attempt: Results from the Army Study to Assess Risk
and Resilience in Servicemembers (Army STARRS). Journal of Abnormal
Psychology.
Onnela, J.-P., & Rauch, S. L. (2016). Harnessing smartphone-based dig-
ital phenotyping to enhance behavioral and mental health. Neu-
ropsychopharmacology, 41(7), 1691–1696. https://doi.org/10.1038/
npp.2016.7
R Core Team. (2016). R: A language and environment for statistical computing.
Vienna, Austria: R Foundation for Statistical Computing. Retrieved from
https://www.R-project.org/
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momen-
tary assessment. Annual Review of Clinical Psychology, 4, 1–32.
https://doi.org/10.1146/annurev.clinpsy.3.022806.091415
Tinbergen, N. (1951). The study of instinct (Vol. xii). New York, NY: Clarendon
Press/Oxford University Press.
Tinbergen, N. (1974). Ethology and stress diseases. Science, 185(4145), 20–
27. https://doi.org/10.1126/science.185.4145.20
Torous, J., Onnela, J.-P., & Keshavan, M. (2017). New dimensions and new
tools to realize the potential of RDoC: Digital phenotyping via smart-
phones and connected devices. Translational Psychiatry, 7(3), e1053.
https://doi.org/10.1038/tp.2017.25
von Neumann, J., Kent, R. H., Bellinson, H. R., & Hart, B. I. (1941). The mean
square successive difference. Annals of Mathematical Statistics, 12(2),
153–162.
Wilson, E. O. (1978). On human nature (Revised ed.). Cambridge, MA: Har-
vard University Press.
Wolff, J. C., Davis, S., Liu, R. T., Cha, C. B., Cheek, S. M., Nestor, B. A., … Spir-
ito, A. (2018). Trajectories of suicidal ideation among adolescents fol-
lowing psychiatric hospitalization. Journal of Abnormal Child Psychology,
46(2), 355–363. https://doi.org/10.1007/s10802-017-0293-6
World Health Organization. (2016a). Monitoring health for the SDGs: Sustain-
able development goals. Geneva: Author.
World Health Organization. (2016b). WHO mortality database. Retrieved
from https://apps.who.int/healthinfo/statistics/mortality/whodpms
How to cite this article: Kleiman EM, Turner BJ, Fedor S, et al.
Digital phenotyping of suicidal thoughts. Depress Anxiety. 2018;
1–8. https://doi.org/10.1002/da.22730