Clinical Psychology Review 95 (2022) 102161

Contents lists available at ScienceDirect

Clinical Psychology Review

journal homepage: www.elsevier.com/locate/clinpsychrev

Review

Linguistic features of suicidal thoughts and behaviors: A systematic review

Stephanie Homan a, b, *, Marion Gabi b, Nina Klee b, Sandro Bachmann c, d, Ann-Marie Moser c, d,
Martina Duri' b, Sofia Michel e, Anna-Marie Bertram f, Anke Maatz a, Guido Seiler c, d,
Elisabeth Stark c, g, Birgit Kleim a, b
a
Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric University Hospital Zurich, University of Zurich, Switzerland
b
Department of Psychology, University of Zurich, Switzerland
c
Zurich Center for Linguistics, University of Zurich, Switzerland
d
German Department, University of Zurich, Switzerland
e
Department of Psychology, University of Fribourg, Switzerland
f
Department of Psychology, University of Basel, Switzerland
g
Institute of Romance Studies, University of Zurich, Switzerland

A R T I C L E I N F O A B S T R A C T

Keywords: Language is a potential source of predictors for suicidal thoughts and behaviors (STBs), as changes in speech
Suicidal thoughts and behavior characteristics, communication habits, and word choice may be indicative of increased suicide risk. We reviewed
Linguistic features the current literature on STBs that investigated linguistic features of spoken and written language. Specifically,
Prosody
we performed a search in linguistic, medical, engineering, and general databases for studies that investigated
Lexicon
First-person singular
linguistic features as potential predictors of STBs published in peer-reviewed journals until the end of November
(morpho)syntax 2021.We included 75 studies that investigated 279,032 individuals with STBs (age = 29.53 ± 10.29, 35% fe­
males). Of those, 34 (45%) focused on lexicon, 20 (27%) on prosody, 15 (20%) on lexicon and first-person
singular, four (5%) on (morpho)syntax, and two (3%) were unspecified. Suicidal thoughts were predicted by
more intensifiers and superlatives, while suicidal behaviors were predicted by greater usage of pronouns,
changes in the amount of verb usage, more prepend and multifunctional words, more nouns and prepositions,
and fewer modifiers and numerals. A diverse field of research currently investigates linguistic predictors of STBs,
and more focus is needed on their specificity for either suicidal thoughts or behaviors.

1. Introduction
Worldwide, a person dies by suicide every 40 s (World Health Or­
ganization, 2020), amounting to 800,000 deaths annually. Death by
suicide was the 18th leading cause of death in 2016 (World Health Or­
ganization, 2020). Previous research has identified suicidal thoughts as
one of the strongest predictors of suicide attempts (Law, Jin, & Anestis,
2018; McHugh, Corderoy, Ryan, Hickie, & Large, 2019; Ribeiro et al.,
2016), but reliable prediction models of suicide risk are lacking. This is
surprising considering the broad body of literature that has investigated
suicidal thoughts in at-risk individuals (Choo, Kuek, & Burton, 2018;
Davidson, Anestis, & Gutierrez, 2017; Kreuze et al., 2017; Lai, Maniam,
Chan, & Ravindran, 2014; Limburg, Watson, Hagger, & Egan, 2017;
Meehan, Mannix, Zafonte, & Pascual-Leone, 2015; Melia et al., 2020;
Notredame et al., 2019; Pospos et al., 2018; Stange, Kleiman, Mermel­
stein, & Trull, 2019; Weisel et al., 2019; Weldon, Poyade, Martin, Sharp,
& Martin, 2019; Witt et al., 2017). Recent studies suggest that self-rating
questionnaires (Mayes et al., 2020) and chart data (Barak-Corren et al.,
2020) might have great predictive power in forecasting suicidal
behavior, while the predictive power of clinician rating scales has been
questioned (Madan et al., 2016; Runeson et al., 2017). The same is true
for predictors of suicidal thoughts and behaviors (STBs). While
numerous risk factors have been suggested (e.g., demographics, hope­
lessness, mental disorders, prior psychiatric treatment, prior STBs,
internalizing psychopathology, and impulsivity) (Franklin et al., 2017;
Klonsky, May, & Saffer, 2016; Nock, Prinstein, & Sterba, 2009), iden­
tifying individuals who have no history of suicide attempts but did
experience suicidal thoughts has not been reliably replicated (Klonsky
et al., 2016). Even prospective predictors based on contemporary
theoretical models, such as the interpersonal theory of suicide (Joiner,
Van Orden, Witte, & Rudd, 2009; Van Orden et al., 2010), only achieved
modest effect sizes in predictive models (Chu et al., 2017). Overall, the

* Corresponding author at: Lenggstrasse 31, 8032 Zurich, Switzerland.

E-mail address: stephanie.homan@bli.uzh.ch (S. Homan).

https://doi.org/10.1016/j.cpr.2022.102161
Received 3 July 2021; Received in revised form 28 March 2022; Accepted 27 April 2022
Available online 6 May 2022
0272-7358/© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

results regarding predictors of STBs are heterogeneous and inconclusive. Table 1

This could be related to unclear definitions. Suicidality is an unspe­ Abbreviations and definitions.
cific term, combining suicidal thoughts, suicide attempt, suicide, and Term Abbreviation Definition
self-injurious behavior (Meyer et al., 2010). Here we will focus on the
Auxiliary verbs/ NA Verbs that add grammatical meaning to a
definitions provided by the US Centers for Disease Control and Pre­ auxiliaries sentence.
vention, which distinguish between self-directed violence (i.e., non- Bi-grams NA A sequence of two adjacent elements
suicidal self-injury) and suicidal self-directed violence. Within the such as letters or words (e.g., ‘good
latter category, suicide is defined as self-directed harmful behavior idea’).
Coherence NA A meaningful connection between the
intended to cause one's death and results in one's death; suicide attempt words in a text.
is a self-directed harmful behavior intended to cause one's death, but Energy entropy NA A measure of abrupt changes (e.g.,
which ultimately results in a nonfatal outcome; and suicidal thoughts voiced sounds represented by lower vs.
are thoughts about, consideration, or planning of a suicide (Crosby, silence by higher entropy).
Formants F0, F1, F2, A distinctive frequency component of the
Ortega, & Melanson, 2011).
etc. acoustic signal (e.g., produced by
speech, music).
1.1. Linguistic features Glottal flow NA The airflow that passes through the vocal
cords at the glottis.
Predictors of STBs that have been under-appreciated are linguistic Jitter NA The measure of fluctuations in the
frequency of vibration of the vocal cords.
features. Language offers a unique window into the human mind and Lingusitic Inquiry and LIWC A text analysis program that counts
individuals' emotional worlds and could thus provide help to identify Word Count tool words in psychologically meaningful
patients at risk for suicide. Clinically, prosodic abnormalities associated categories.
with an individual's mental state have been well documented since Machine learning ML The science that aims at getting
computer algorithms to improve
Kraepelin (1921), one of psychiatry's founding fathers who described a
automatically through experience.
pre-suicidal voice as ‘hollow’ and ‘toneless.’ Changes in the acoustic Mel Frequency MFCC The coefficients that describe loudness
characteristics of the voice of individuals in distress have been attributed Cepstral Coefficient and pitch of the human auditory system.
to a complex interaction between the muscular system and cognitive Morphology NA Examines the structure of a single word.
processes that are influenced by even the slightest shift in affective states (Morpho)syntax NA The structure of singles words and rules
with which sentences are made up of
(K. Scherer, 1986). These acoustic changes can be captured using pro­ individual words.
sodic, articulatory, and acoustic speech features. Silverman, Silverman, Natural language NLP The use of computational methods to
et al. (2006) noticed discernible changes in patients' voices during a shift processing analyze and “understand” natural
to a pre-suicidal state in recorded therapy sessions to hollow and tone­ language and speech.
Peak slope NA A speech signal used to distinguish
less sounds, intensity loss during utterances, monotonous, mechanical,
between breathy, modal, and tense voice
and repetitious phrasing, and unexpected emphasis unrelated to normal qualities.
syntax (for a definition of the linguistic terms used in this article see Phrasal nodes NA A phrasal node is e.g., an article plus
Table 1). Subsequently, several additional acoustic features have been adjective plus noun (‘the young
identified and investigated. woman’).
Pitch NA Refers to the perceived sound of F0
(fundamental frequency of the vocal
1.2. Prosodic features cords).
Power Spectral PSD A measure of a signal's power intensity
Speech analysis has revealed that individuals with STBs exhibit Density (loudness) by frequency.
Probability density NA Estimated by computing histograms
prosodic alterations such as unusual and unexpected vocal patterns,
function based on the voiced/unvoiced/silence
changes in acoustic properties, and shifts in spectral energy (Larsen, labeling of frames.
Frandsen, & Erlangsen, 2016), although this was not reported consis­ Subordination NA A linkage of at least two sentences,
tently (Cummins et al., 2015). Of the aggregated studies from these two where one is syntactically integrated into
reviews only one was a controlled, prospective study (S. Scherer, Pes­ the other (subordinate).
Syntax NA Examines the rules according to which
tian, & Morency, 2013), while the others investigated a single group of sentences are made up of individual
individuals with suicidal thoughts. Further, source features character­ words.
izing voice production (i.e., the path that the air travels from the lungs Uni-grams NA Sequence of one element such as one
through the glottis) have also been indicated as important aspects of letter or word (e.g., ‘idea’).
suicidality. Particularly, glottal flow and jitter (i.e., variations in glottal Note. The table gives an overview of used (linguistic) terms and contains a short
pulse timing during articulation), in addition to other voice quality definition of the respective term as well as an abbreviation of the term. If no abbre­
features (“[…] the auditory perception of changes in vocal fold vibration viation is used in the article, we have inserted “NA” for “not available/not necessary”
and vocal tract shape, outside of pitch, loudness, and phonetic cate­ in the column “abbreviation”.
gory.” p. 24, Cummins et al., 2015), have been associated with suici­
dality in controlled studies (Ozdas, Shiavi, Silverman, Silverman, & Wilkes, Shiavi, & Salomon, 2007, Yingthawornsuk & Shiavi, 2008). No
Wilkes, 2004; S. Scherer et al., 2013). Another important linguistic study has reported associations of MFCC, the coefficients that describe
feature might be formants that convey information about the acoustic the loudness and pitch of the human auditory system, with STBs. Taken
resonances of the vocal tract (France, Shiavi, Silverman, Silverman, & together, there is a rich pool of linguistic features that might include
Wilkes, 2000). However, they have not been investigated in a controlled suitable predictors of STBs. Besides these linguistic features, syntax
study. Finally, spectral features that characterize the speech signal's might provide additional useful features for reliably predicting STBs.
frequency distribution at a certain time - Power Spectral Density (PSD)
and Mel Frequency Cepstral Features (MFCC) (Cummins et al., 2015) -
have been investigated with homogeneous results. In some studies, the 1.3. (Morpho)syntactic features
shift in PSD, a measure of a signal's intensity by frequency (i.e., loud­
ness) progressed from lower to higher frequencies (France et al., 2000; One central class of syntactic features is concerned with information
Ozdas et al., 2004), while it progressed in the opposite direction in other structure and packaging in utterances (Givón, 1983; Vallduvi, 1992).
studies (Yingthawornsuk et al., 2006; Yingthawornsuk, Keskinpala, Information structure is a well-studied area in linguistics (Altmann,

2
S. Homan et al.
1981; Krifka, 2007; Musan, 2002) and refers to aspects of spoken lan­
guage that speakers use to transmit information relative to the ad­
dressee's current mental state (Krifka, 2007). That is, the speaker
structures the speech production so that a common understanding can
emerge between speaker and listener, and important pieces of infor­
mation are highlighted to promote mutual understanding and coherence
(Krifka, 2007). Functional units of information structure include topics
(‘what the utterance is about’), closed or open focus (‘the most impor­
tant element in an utterance to make the utterance true’), givenness, and
frame setting (‘the domain/time/place where the situation described in
the utterance is valid’). Various phenomena indexed in the context of
information packaging (see the overview by Musan, 2002) all presup­
pose the capacity of speakers to correctly calculate the state of knowl­
edge of the listener, to adequately accommodate their utterances and to
communicate effectively.
Another central class of syntactic features relates to syntactic
complexity, which can be measured based on sentence length or the
amount of subordination (Szmrecsanyi, 2004). More phrasal nodes of a
unit of spoken or written language reflect a more complex syntactic
structure. Thus, the number of nodes provides a formal and psycho­
logical characterization of the speaker's ability to construct and produce
sentences that conform to the syntactic rules of their language. Next,
word count gives a value for the sentence length and thus provides a
proxy for syntactic complexity. Last, the index of syntactic complexity
refers to “[…] a given unit of data [that] is twice the number of occur­
rences of subordinating conjunctions and WH-pronouns [who, whose,
whom, which] plus the number of occurrences of verb forms and noun
phrases in that unit” (p. 1035, Szmrecsanyi, 2004). Recently, complexity
has become a focus of linguistic theory (Culicover, 2014; Givón & Shi­
batani, 2009; Szmrecsanyi, 2004; Trotzke & Bayer, 2015), which has
also been examined in studies of depression (Esmaeelpour & Sasani,
2018) and executive functions (Balaban & Hohenberger, 2020; Delage &
Frauenfelder, 2019; Fernandez-Duque, 2009). In depression, for
instance, findings suggest that acutely depressed individuals tend to
produce speech with less syntactic complexity.
1.4. Sources of linguistic features
Identifying markers of both written and spoken language has
received increasing interest with the emergence of machine learning
(ML) and natural language processing (NLP), although there are new
challenges as well. For example, algorithms developed to extract lin­
guistic features from clinicians' notes might only work optimally with a
particular text source and may therefore not be easily transferable to
other text sources (Demner-Fushman, Chapman, & McDonald, 2009;
Kreimeyer et al., 2017). Despite such challenges, information shared on
social media platforms provides another valuable and widely available
data source for suicidality predictors (Notredame et al., 2019). In
addition to written language, the acoustic features of spoken language
offer another potential source for STBs (Cummins et al., 2015; Larsen
et al., 2016). With technological advances, mobile applications provide
a spectrum of possibilities for collecting data on acoustic features and
information delivery (Larsen et al., 2016). Provided the respective fea­
tures show replicable predictive strength, a potential future avenue for
analyzed acoustic features of speech patterns, provided further repli­
cation of selected key features, could be their use in algorithms for just-
in-time adaptive interventions (Kleiman, Glenn, & Liu, 2019) that could
deliver treatment and help at-risk individuals.
We systematically reviewed the current literature published between
2000 and 2021 investigating language in individuals identified as at risk
for STBs. We included studies on individuals endorsing suicidal
thoughts, individuals who previously attempted suicide, individuals
who endorsed suicidal thoughts and previously attempted suicide, in
addition to studies on those who died by suicide. We sought to sum­
marize the current evidence for linguistic features as predictors of STBs.
Then, from this evidence, we aimed to identify the most promising cross-
3
Clinical Psychology Review 95 (2022) 102161
sectional and longitudinal linguistic features based on their predictive
power in forecasting STBs.
2. Methods
2.1. Search strategy
We performed an electronic search of the PubMed database, Psy­
chInfo, Web of Science, EBESCO, Linguistics and Language Behavior
Abstracts, Modern Language Association International Bibliography,
and Cogprints including studies through the end of November 2021. We
used the following search terms “suicidal thoughts” OR “suicidal
behavior” OR suicid* AND linguistic OR “speech markers” OR “language
markers” OR speech* OR “speech features” OR “speech signal analysis”
OR “voice analysis” OR “text mining” OR “natural language processing,”
with an asterisk allowing articles with derivations of a key word to be
included in the search results. Our searches included journal articles
published in English that included human subjects. We excluded meta-
analyses, systematic reviews, other types of reviews, case reports,
books, and documents. In addition to the electronic database search, we
searched the references of previous reviews and meta-analyses (Cum­
mins et al., 2015; Demner-Fushman et al., 2009; Franklin et al., 2017;
Larsen et al., 2016; Notredame et al., 2019). We used the cross-reference
technique to identify additional relevant papers by reviewing papers
that cited articles in which linguistic features in individuals with STBs
appeared (i.e., forward search). Further, we reviewed articles that were
cited by articles meeting our selection criteria (i.e., backward search) or
that were cited in major review articles and chapters on linguistic fea­
tures (i.e., forward search).
2.2. Selection criteria
We included studies that met the following inclusion criteria: (1)
investigated individuals experiencing suicidal thoughts or who
attempted suicide or died by suicide, (2) investigated linguistic features,
(3) assessed non-lyrical written or spoken language, (4) cross-sectional
and prospective studies, (5) full-text available, and (6) published in
English. Opinion papers, case reports and reviews were excluded.
We adhered to the PRISMA guidelines (Moher et al., 2015). Inde­
pendent researchers (M.G., N.K., M.D., S.M., and A-M.B.) conducted the
literature search, screened the articles, eliminated duplicates, decided
whether articles met the inclusion criteria, and extracted all relevant
data from the final articles. Their independent searches were compared
by S.H., and discrepancies were investigated and resolved by discussion
with S.H.
We recorded the following variables: country of study origin, type of
study design, sample size, characteristics of the study's population (age,
sex, psychiatric diagnosis), symptoms rated by a clinically trained or
non-clinically trained rater, number and type of data points analyzed,
and means with which language was assessed (vocal exercise, read
speech, free response or interview, free speech, or written text).
Regarding the category “rater,” we considered “clinically trained” as a
trained clinical psychologist or psychiatrist who rated either patients'
symptoms or judged texts as “with STB content.” Further, we catego­
rized studies based on investigated the linguistic features into (1)
“prosody” (features related to voice, pitch, speech rate, pronunciation,
hesitations, and speech errors), (2) “lexicon” (the occurrence of specific
words such as ‘fear’ and ‘dark’), (3) “first-person singular pronouns” (i.
e., overuse of first-person singular pronouns including ‘I,’ ‘me,’ ‘mine’),
(4) “(morpho)syntax” (length and syntactic complexity of the sentences
uttered by the patient, word order variability, tense), and (5) “unspec­
ified” (e.g., studies with a broader interest in language without explicitly
or implicitly specifying specific linguistic categories, e.g., only ‘speech’
is mentioned).
S. Homan et al.
2.3. Data and code availability
This paper was written using RStudio (version 1.4.1106) (RStudio
Team, 2020), R packages include rmarkdown (version r 2.11); repre­
search (version r 0.0.0.9000; https://github.com/phoman/represearch/
); knitr (version r 1.36); and papaja (version r 0.1.0.9997). All data and
code are freely available online to ensure reproducibility (https://osf.
io/f3u7s/). This review is preregistered at PROSPERO (ID:
CRD42020221730).
3. Results
3.1. Descriptive statistics
We identified 709 potential studies, 162 of which were screened
carefully for eligibility. A total of 75 studies met the inclusion criteria
(Fig. 1).
We included 75 studies published between 2000 and 2021, with a
total of 279,032 patients (mean age = 29.53 ± 10.29 years, range =
Fig. 1. PRISMA flow chart. In adherence with the PRISMA guidelines (Moher et al., 2015), independent researchers (M.G., N.K., M.D., S.M., and A.-M.B.) conducted
the literature search and extracted the data. Their searches were compared by S.H.
4
Clinical Psychology Review 95 (2022) 102161
12.80, 57.50), including 98,097 females (35%). Information on the
investigated samples' age and sex was not available for 32 studies (43%).
Approximately half of the studies reported data from clinical pop­
ulations while the other half reported data from non-clinical pop­
ulations. The latter included social media users, forum users, students,
and veterans. The studies that investigated deaths by suicide were also
categorized as “non-clinical” when no psychiatric treatment history was
mentioned. Patients experienced suicidal thoughts (24, 32%), had
attempted suicide (11, 15%), or both (10, 13%), had STBs (not otherwise
specified 22, 29%), or had died by suicide including euthanasia (7,
11%).
See Tables 2–6 for more details on sample descriptions stratified by
studies on prosody, lexicon, or other linguistic features, respectively. For
most studies it was possible to determine whether the focus was on
suicidal thoughts or behaviors (summarized in the respective categories
“suicidal thoughts” or “suicidal behaviors”). For all other studies that
used the term suicidality or suicide risk, the findings were summarized in
the more general category “STBs”.
The linguistic features investigated in the studies fall into the
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

Table 2
Study and sample characteristics for studies investigating prosody in language of individuals with STBs.
Study Design Country N Sample Symptom Assessed Rater Age (M) Age (SD)

Belouali et al., 2021 LON USA 124.00 Veterans ST Self-report psychatric scales CLIN NA NA
Bryan et al., 2018 OBS USA 54.00 Active duty soldiers ST Structured interview CLIN NA NA
Cohn et al., 2009 RCT USA 57.00 Psychiatric pat. ST HDRS CLIN NA NA
Figueroa Saavedra et al., 2020 Pilot Chile 60.00 University students Suicidal OSS N-CLIN 18.62 0.49
France et al., 2000 OBS USA 67.00 Psychiatric male pat. SB SA or fixation on suicide CLIN NA NA
Gideon et al., 2019 CROSS-S USA 43.00 Psychiatric pat. STBs Asked for STBs CLIN NA NA
Hashim et al., 2012 OBS USA 19.00 Psychiatric pat. Suicidal NA N-CLIN NA NA
Hashim et al., 2017 PROS USA 116.00 Psychiatric pat. ST HDRS CLIN 43.30 10.20
Keskinpala et al., 2007 OBS USA 97.00 Psychiatric pat. Suicidal Categorized by a clinician CLIN 45.00 NA
Landau et al., 2007 OBS USA 107.00 Psychiatric pat. Suicidal BDI CLIN NA NA
Ozdas et al., 2001 OBS USA 30.00 Psychiatric pat. Suicidal STBs N-CLIN NA NA
Ozdas et al., 2004 CON USA 30.00 Psychiatric pat. Suicidal STBs CLIN NA NA
Pestian et al., 2016 CON USA 60.00 Psychiatric pat. STB CSSRS, SIQ CLIN NA 30.00
Pestian et al., 2017 CON USA 371.00 Psychiatric pat. ST CSSRS N-CLIN 33.60 16.40
Scherer et al., 2013 CON USA 16.00 Psychiatric pat. STB CSSRS N-CLIN 15.53 2.12
Stasak et al., 2021 CON Australia 246.00 Psychiatric pat. STB Suicide-specific database N-CLIN NA NA
Yingthawornsuk et al., 2006 OBS USA 32.00 Psychiatric male pat. Suicidal BDI CLIN NA NA
Yingthawornsuk et al., 2007 OBS USA 20.00 Psychiatric female pat. Suicidal BDI CLIN 57.50 NA
Yingthawornsuk & Shiavi, 2008 OBS USA NA Psychiatric male pat. Suicidal BDI CLIN NA NA
Zhang et al., 2020 CROSS-S USA 222.00 MH website users Suicidal PHQ-9 N-CLIN NA NA

Note. BDI, Beck Depression Inventory; CAMS, Collaborative Assessment and Management of Suicidality; CLIN, clinically trained rater; CON, controlled study; CROSS-S,
cross-sectional study; CSSRS, Columbia-Suicide Severity Rating Scale; EXP, exploratory study; HDRS, Hamilton Rating Scale for Depression; LON, longitudinal study;
N, sample size; N-CLIN, not clinically trained rater; NA, not available; OBS, observational study; OSS, Okasha's Suicidality Scale; Pat., patients; PHQ-9, Patient Health
Questionnaire 9-Item; Pilot, pilot study; PROS, prospective study; Rater, symptoms/text rated by a clinically trained rater (CLIN) or non-clinically trained rater (N-
CLIN); RCT, randomized controlled trial; SA, suicide attempt; SB, suicidal behavior; SIQ, Suicidal Ideation Questionnaire; SPS, Suicide Probability Scale; ST, suicidal
thoughts; STB, suicidal thoughts and behaviors; USA, United States of America.

following categories and used various means to acquire those features
(Fig. 2):
- Prosody/phonetics (27%): Almost one third of the studies focused on
prosodic features with some studies investigating other phonetic
properties only loosely related to prosody (e.g., speech transmission
and reception).
- Lexicon (45%): Almost half of the studies investigated lexical
features.
- Lexicon, first-person singular (20%): Several studies investigated
first-person pronouns together with other lexical features.
- Lexicon, (morpho)syntax (5%): Four studies investigated syntactic
features together with lexicon.
- Unspecified (3%): Two studies fell into this category.
For more information on linguistic features see Tables 7–11.
with ML and NLP to screen U.S. veterans who experienced suicidal
thoughts. Veterans answered the clinician-administered Patient Health
Questionnaire (PHQ-9), particularly item 9, which evaluates passive
thoughts of death or self-injury. Veterans were audio recorded at every
assessment point (6 points in total). The speech of veterans with suicidal
thoughts was flatter and less animated in tone (e.g., a skewness closer to
zero), more monotonous and duller (e.g., shown by different indices of
the MFCC), less energetic and filled with more abrupt changes (less
energy entropy and lower voiced tilt), and breathier (greater variability
between glottal closures) compared with veterans without suicidal
thoughts. Further, veterans experiencing suicidal thoughts used more
possessive pronouns (e.g., ‘my,’ ‘his,’ ‘hers’), superlative adverbs, and
family references (e.g., ‘daughter,’ ‘aunt,’ ‘dad’). The unique combina­
tion of a subset of linguistic (superlative adverbs, possessive pronouns,
proper nouns, agentic language) and acoustic features (energy dy­
namics, MFCC, F0, glottal flow indices) best discriminated the recordings
of veterans with verses without STBs.

3.2. Prosody 3.2.2. Suicidal behavior

Overall, 20 studies investigated prosody in terms of voice, pitch,
speech rate, pronunciation, hesitations, or speech errors in relation to
STBs (Table 7).
3.2.1. Suicidal thoughts
3.2.1.1. Free response or interview. Bryan et al. (2018) examined the
emotional bond between clinicians and their patients, who were active-
duty soldiers with STBs, by analyzing the vocally encoded emotional
arousal as measured by mean fundamental frequency (F0), the frequency
at which vocal cords vibrate in voiced sounds. Patients underwent a
structured interview with a clinician for assessing suicidal thoughts. The
mean F0 of patients and clinicians co-varied over time, indicating that
the emotional arousal of patients was at least moderately matched by
clinicians. This is promising as this vocal therapeutic synchronicity
seems to be related to the therapeutic emotional bond, which in turn is
predictive of therapy outcome in terms of suicidal thoughts.
3.2.1.2. Speech recordings. Belouali et al. (2021) used prosodic features
3.2.2.1. Free response or interview. Ozdas, Shiavi, Wilkes, Silverman,
and Silverman (2001) investigated vocal-cord jitter and the glottal flow
spectrum to differentiate between individuals with a history of suicide
attempts, patients with a diagnosis of depression, and healthy controls.
Individuals with a history of suicide attempts and patients with a diag­
nosis of depression had increased vocal-cord jitter and glottal spectral
slope compared with controls.
3.2.3. STBs
3.2.3.1. Free response or interview. Response latency and F0 variability
were investigated by Cohn et al. (2009). In a psychiatric population
assessed with the clinician-administered Hamilton Rating Scale for
Depression (HDRS) regarding suicidal thoughts, lower F0, and less
speaker-switch duration (the prolonged response time associated with
depression) were predictive of STB treatment success. Gideon, Schatten,
McInnis, and Provost (2019) examined phone conversations of patients
with STBs who had been assessed by a trained clinician. Conversations

5
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

Table 3
Study and sample characteristics for studies investigating lexicon in language of individuals with STBs.
Study Design Country N Sample Symptom Assessed Rater Age Age
(M) (SD)

Adam-Troian & Arciszewski, OBS USA NA Google search engine Fatal Search query data N- NA NA
2020 users suicide CLIN
Adamou et al., 2019 Pilot UK 130.00 Persons who died by Fatal death Suicide-specific CLIN 44.91 15.93
suicide database
Aladağ et al., 2018 POC USA NA Forum users ST Suicide-specific forum CLIN NA NA
Al-Mosaiwi & Johnstone, OBS UK 2741.00 Forum users ST Suicide-specific forum CLIN NA NA
2018
Bantilan et al., 2020 CROSS- USA 1864.00 Telemedcin platform STB Categorized by rater CLIN NA NA
S users
Braithwaite et al., 2016 OBS USA 135.00 Social media users Suicidal DSI - Suicide Subscale N- NA NA
CLIN
Cheng et al., 2017 EXP China 976.00 Social media users Suicidal SPS N- NA NA
CLIN
Cohen, 2016 CROSS- USA 211.00 Suicide bombers Fatal death Suicide notes CLIN NA NA
S
Cohen et al., 2020 Pilot USA 60.00 Adolescent pat. Suicidal PHQ-A, clinical CLIN 12.80 2.40
impression
Coppersmith et al., 2016 EXP USA 554.00 Social media users SB Survived a suicide CLIN NA NA
attempt
Coppersmith et al., 2018 CROSS- USA 418.00 Social media users SB SB in the next 6 month CLIN 22.00 NA
S
Dieltjens et al., 2014 Pilot Netherlands 5.00 Suicidal pat. Fatal Request for euthanasia CLIN NA NA
suicide
Fahey et al., 2020 EXP Japan NA Social media users STB Keyword search of posts CLIN NA NA
Fernandes et al., 2018 OBS UK NA Psychiatric pat. STB Keyword search of posts CLIN NA NA
Glenn et al., 2020 Pilot USA 33.00 Students with SA STB Asked for STBs N- 20.40 2.40
CLIN
Grant et al., 2018 CROSS- USA 63,252.00 Social media users ST Suicide-specific forum N- NA NA
S CLIN
Grundlingh, 2018 CROSS- South Africa 99.00 Males who died by Fatal Died by suicide N- NA NA
S suicide suicide CLIN
Homan et al., 2014 OBS USA 6237.00 Social media users Suicidal Keyword search of posts CLIN NA NA
Howard et al., 2020 CROSS- Canada NA Social media users ST Suicide-specific forum CLIN NA NA
S
Ji et al., 2018 CROSS- Australia NA Social media users ST Suicide-specific forum N- NA NA
S CLIN
Lester & Leenaars, 2016 PROS USA 679.00 Persons who died by Fatal Died by suicide CLIN NA NA
suicide suicide
Levis et al., 2021 CON USA 246.00 Veterans with PTSD Fatal death Clinician notes CLIN NA NA
Litvinova et al., 2017 CROSS- Russia 1045.00 Forum users Fatal Died by suicide N- NA NA
S suicide CLIN
Liu et al., 2019 CROSS- China NA Social media users ST Keyword search of CLIN NA NA
S blogpost
Lv et al., 2015 CROSS- China 1031.00 Social media users STB Died by suicide CLIN NA NA
S
Nguyen et al., 2017 CROSS- Australia NA Forum users ST Labeled profils N- NA NA
S CLIN
O'Dea et al., 2015 OBS Australia NA Social media users Suicidal Keyword search of posts CLIN NA NA
Ren et al., 2015 OBS Japan NA Social media users STB Information from blogs N- NA NA
CLIN
Sawhney et al., 2021 OBS India 500.00 Social media users STB CSSRS CLIN NA NA
Song et al., 2016 CROSS- South Korea NA Social media users ST Keyword search of posts N- NA NA
S CLIN
Tadesse et al., 2020 CROSS- China NA Social media users ST Suicide-specific forum N- NA NA
S CLIN
Vioules et al., 2018 CROSS- France 60.00 Social media users ST Clinician impression CLIN NA NA
S
Xu et al., 2021 OBS Hong Kong 5682.00 MH website users Suicidal Categorized by a CLIN NA NA
counselor
Zhong et al., 2018 CROSS- USA 682.00 Pregnant women SB ICD codes CLIN NA NA
S

Note. CAMS, Collaborative Assessment and Management of Suicidality; CLIN, clinically trained rater; CON, controlled study; CROSS, cross-over study; CROSS-S, cross-
sectional study; CSSRS, Columbia-Suicide Severity Rating Scale; DSI, Depressive Symptom Inventory; EXP, experimental study; N, sample size; N-CLIN, non-clinically
trained rater; NA, not available; ICD, International Classification of Disease; OBS, observational study; Pat., patients; Pilot, pilot study; PHQ-A, Patient Health
Questionnaire 9-Item Modified for Adolescents; POC, proof-of-concept study; PROS, prospective study; PTSD, Post-traumatic stress disorder; Rater, symptoms/text
rated by a clinically trained rater (CLIN) or non-clinically trained rater (N-CLIN); SA, suicide attempt; SB, suicidal behavior; ST, suicidal thoughts; STBs, suicidal
thoughts and behaviors; SPS, Suicide Probability Scale; UK, United Kingdom; USA, United States of America.

of individuals with STBs exhibited lower levels of emotion variability STBs. Increased mean vocal jitter was useful for distinguishing patients
compared with other patients. It is not clear whether certain linguistic with STBs from healthy controls, while an increased spectral slope
features contributed to this result. Ozdas et al. (2004) analyzed jitter and measurement of the glottal flow was useful for distinguishing patients
glottal flow in a psychiatric population assessed by a trained clinician for with STBs from patients with a diagnosis of depression and no STBs, and

6
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

Table 4
Study and sample characteristics for studies investigating first-person singular & lexicon in language of individuals with STBs.
Study Design Country N Sample Symptom Assessed Rater Age (M) Age (SD)

Brancu et al., 2016 Pilot USA 148.00 University students Suicidal CAMS CLIN 21.40 3.82
Brown et al., 2019 OBS Germany 52.00 Social media users ST Semi-structured interview CLIN 16.60 0.96
De Choudhury et al., 2016 OBS USA 440.00 Social media users ST Suicide-specific forum N- NA NA
CLIN
Cook et al., 2016 OBS Spain 1453.00 Psychiatric pat. ST Discharged after self-harm CLIN 40.50 19.59
Kim et al., 2019 PROS Korea 112.00 Persons who died by suicide Fatal suicide Died by suicide CLIN NA NA
Leavitt et al., 2021 OBS USA 114.00 Persons who died by suicide STBs Specific platform CLIN NA NA
Lekkas et al., 2021 OBS USA 52.00 Social media users ST Questions CLIN 16.60 NA
Li et al., 2021 OBS China NA Forum users Suicidal Suicide-specific forum CLIN NA NA
Nobles et al., 2018 CROSS USA 26.00 University students STB Asked for STBs CLIN 20.42 2.55
O'Dea et al., 2017 OBS Australia NA Social media users Suicidal Keyword search of posts CLIN NA NA
O'Dea et al., 2021 LON Australia 38.00 Social media users ST PHQ-9 CLIN 29.50 10.30
Pestian et al., 2008 OBS USA 66.00 Psychiatric pat. SB Died by suicide N- NA NA
CLIN
Pestian et al., 2020 CON USA 253.00 Psychiatric pat. Suicidal CSSRS N- 33.50 23.35
CLIN
Sierra et al., 2021 OBS Mexico 582.00 Social media users Suicidal Keyword search of posts CLIN NA NA
Zhang et al., 2021 CROSS- USA 3090.00 Prisoners Fatal suicide Fatal suicide N- NA NA
S CLIN

Note. CAMS, Collaborative Assessment and Management of Suicidality; CLIN, clinically trained rater; CON, controlled study; CROSS, cross-over study; CROSS-S, cross-
sectional study; CSSRS, Columbia-Suicide Severity Rating Scale; DSI, Depressive Symptom Inventory; LON, longitudinal study; N, sample size; N-CLIN, non-clinically
trained rater; NA, not available; OBS, observational study; Pat., patients; PHQ-9, Patient Health Questionnaire 9-Item; Pilot, pilot study; POC, proof-of concept study;
PROS, prospective study; Rater, symptoms/text rated by a clinically trained rater (CLIN) or non-clinically trained rater (N-CLIN); SB, suicidal behaviors; ST, suicidal
thoughts; STBs, suicidal thoughts and behaviors.

Table 5
Study and sample characteristics for studies investigating lexical and (morpho)syntactic indices in language of individuals with STBs.
Study Design Country N Sample Symptom Assessed Rater Age Age
(M) (SD)

Desmet & Hoste, 2018 CROSS- Netherlands NA Social media users ST Keyword search of CLIN NA NA
S posts
Fernández-Cabana et al., CROSS- Spain 23.00 Person who died by suicide Fatal death Suicide notes CLIN NA NA
2015 S
Lien & Zhang, 2020 CROSS- USA 183,993.00 Surveyed individuals ST Suicide-specific CLIN 41.40 16.00
S database
Teixeira et al., 2021 CROSS- Portugal 139.00 Persons who died by ST Died by suicide CLIN NA NA
S suicide

Note. CLIN, clinically trained rater; CROSS-S, cross-sectional study; N, sample size; NA, not available; Pat., patients; Rater, symptoms/text rated by a clinically trained
rater (CLIN) or non-clinically trained rater (N-CLIN); ST, suicidal thoughts; USA, United States of America.

Table 6
Study and sample characteristics for studies investigating unspecified linguistic indices in language of individuals with STBs.
Study Design Country N Sample Symptom Assessed Rater Age (M) Age (SD)

Audenaert et al., 2002 OBS Belgium 20.00 Psychiatric pat. SB Recent SA (<7 days) CLIN 27.55 8.17
Galatzer-Levy et al., 2021 LON Switzerland 20.00 Psychiatric pat. ST BSS CLIN NA NA

Note. BSS, Beck Scale for Suicide Ideation; CLIN, clinically trained rater; LON, longitudinal study; N, sample size; OBS, observational study; Pat., patients; Rater,
symptoms/text rated by a clinically trained rater (CLIN) or non-clinically trained rater (N-CLIN); SA, suicide attempt; SB, suicidal behaviors; ST, suicidal thoughts.

controls. In two studies, Pestian et al. (2016, 2017) investigated acoustic
features of the speech of adolescents with STBs, as determined by the
clinician-administered Columbia Suicide Severity Rating Scale (CSSRS)
and Suicidal Ideation Questionnaire (Pestian et al., 2016), with the
CSSRS administered by a researcher not clinically trained (Pestian et al.,
2017). Their first study (Pestian et al., 2016) included recorded con­
versations of 30 individuals with STBs and 30 matched controls. Using
NLP and ML, 97% of subjects were correctly classified. Yet, it is unclear
which linguistic features NLP focused on, as they only mentioned that
the number of unique words was greater in individuals with STBs. In
their second study (Pestian et al., 2017), adolescents were categorized
into one of three groups based on their CSSRS scores (“suicidal,”
“mentally ill but not suicidal,” or “healthy controls”). Linguistic (e.g.,
word instances and word-pair instances) and acoustic features (e.g., F0,
maximum dispersion quotient, peak slope, pause lengths characterized
by formants F1-F5) were used in an ML model. The results showed that
the combination of linguistic and acoustic features resulted in the best
classification compared to linguistic features alone and acoustic features
alone. However, no information was given regarding which linguistic
features were extracted and which features in general contributed to this
result.
3.2.3.2. Read speech. Hashim, Wilkes, Salomon, and Meggs (2012)
investigated the timing patterns (i.e., the relationship between and
characteristics of voiced, unvoiced and silence frames) underlying the
speech of psychiatric patients with STBs. However, no information was
given on how the researchers determined STBs. The timing patterns
proved valuable for classifying patients as “high risk for suicide” or
“depression.” In a subsequent study (Hashim, Wilkes, Salomon, Meggs,
& France, 2017), the authors used a combination of spectral and timing
measures obtained from the read speech of psychiatric patients assessed
with the clinician administered HDRS to predict suicide risk and

7
S. Homan et al.
Fig. 2. Linguistic features. The bar plot shows the linguistic categories that were investigated by the included studies (‘Lexicon’, ‘Lexicon, first-person singular’,
‘Lexicon, (morpho)syntax’, ‘Prosody’, and ‘Unspecified’). The linguistic features were assessed using the following techniques: four times using vocal exercise, 10
times using read speech, 20 times using free response or an interview, 13 times using free speech, and 56 times using written text.
depression severity. The 67 extracted acoustic features (e.g., indices of
PSD, MFCC, and transition) effectively predicted suicide risk in both
sexes, and depression severity in males. Stasak et al. (2021) examined
audio recordings from the Butler-Brown Read Speech database, which
contains data on inpatients assessed by trained clinicians for psycho­
pathology. Individuals with STBs hesitated more and made more speech
errors compared to the controls. Additionally, the voices of individuals
with STBs had poorer average subjective voice quality (e.g., ‘grade’ and
‘roughness’) than the controls.
3.2.3.3. Read speech, free response, or interview. Figueroa Saavedra et al.
(2020) used voice and speech samples from university students who had
been assessed by a researcher not clinically trained with the Okasha's
Suicidality Scale for STBs. Lower F0, formants (F1, F2) and jitter values
significantly predicted suicidal thoughts in both sexes. An early study by
France et al. (2000) investigated males with STBs assessed by a clinically
trained rater. The authors used speech samples from audio recordings to
examine voice fundamental frequency (F0), amplitude modulation,
formants (F1, F2, F3), and power distribution (percentages of the total
power in four subbands from 500 to 2000 Hz; PSD1− 4). Formants F1 and
F3, skewness of amplitude modulation, along with PSD2 best discrimi­
nated individuals with STBs from patients diagnosed with depression
and healthy controls. Interestingly, features describing F0 were ineffec­
tive discriminators. Keskinpala, Yingthawornsuk, Wilkes, Shiavi, and
Salomon (2007) examined MFCC and PSD in a psychiatric population
assessed by a trained clinician for STBs. Successful classification of
depressed patients with and without STBs was achieved with both MFCC
and PSD. The latter was better for classifying female voices. Landau,
Yingthawornsuk, Wilkes, Shiavi, and Salomon (2007) investigated
speech in a psychiatric population assessed with the clinician-
administered Beck Depression Inventory (BDI) regarding PSD. The
percentage of power in the frequency bands (i.e., four 500 Hz subbands
covering 0–1500 Hz) predicted who was having STBs, was depressed, or
remitted. Interestingly, spontaneous speech was easier to model. In line
with France et al. (2000) findings, Scherer et al. (2013) found no value
of F0 in distinguishing psychiatric adolescents with STBs from those
without, assessed with the CSSRS by a non-clinically trained rater.
However, smaller values of glottal flow indices, such as normalized
amplitude quotient (a direct measure of glottal flow) and peak slope (a
speech signal used to distinguish between breathy, modal, and tense
8
Clinical Psychology Review 95 (2022) 102161
voice qualities), were associated with STBs. Further, the voices of ado­
lescents with STBs were breathier than those without (i.e., more sym­
metric glottal pulses and lower frequency of the glottal formant). The
pre-suicidal mental state seemed to produce changes in the mecha­
nisms of speech production, thus altering those voice features. Yingth­
awornsuk et al. (2006) and Yingthawornsuk & Shiavi (2008)
investigated three groups of male psychiatric patients: at-risk patients
for STBs, patients diagnosed with depression, and remitted depression.
All were assessed by a clinically trained rater. Speech samples were
collected from a therapy session (‘free speech’) and from reading
(‘structured speech’). In their first study (Yingthawornsuk et al., 2006),
features of vocal output such as PSD, peak power, and frequency of the
peak power, were useful for differentiating the readings of at-risk pa­
tients from those with a diagnosis of depression but no STBs. In their
second study (Yingthawornsuk & Shiavi, 2008), spontaneous speech was
used successfully (with PSD in four ranges: 0–0.5 KHz, 0.5-1KHz, 1–1.5
KHz, and 1.5–2 KHz, and measures for pitch periodicity) to distinguish
between the group of at-risk patients and patients diagnosed with
depression. The same group published data on a female psychiatric
sample who were also assessed by a trained clinician for STBs (Yingth­
awornsuk et al., 2007). PSD and bandwidth, as opposed to the center
frequency and weight coefficient, were again powerful in distinguishing
the speech of patients with STBs from that of patients with a depression
diagnosis but no STBs.
3.3. Lexicon
Thirty-four studies investigated prosody in terms of the occurrence of
specific words in relation to STBs (Table 8).
3.3.1. Suicidal thoughts
3.3.1.1. Health records. Fernandes et al. (2018) trained an ML algo­
rithm to search electronic health records established by clinicians for
mentions of ‘suicidal thoughts’ or ‘suicide attempt.’ While the algorithm
successfully identified patients experiencing suicidal thoughts based on
their health records, it remains unclear whether reliable predictions
about future STBs are possible with such an approach.
3.3.1.2. Forum posts. Aladağ, Muderrisoglu, Akbas, Zahmacioglu, and
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

Table 7 Table 8
Liguistic feature descriptives of the prosody-focused studies. Liguistic feature descriptives of the lexicon-focused studies.
Study # Datapoint LIWC Aim Study # Datapoints Datapoint LIWC Aim
Datapoints type type

Belouali et al., 588.00 Audio yes Identify ST Adam-Troian & 700.00 Search query no Predict suicide
2021 recordings linguistic features Arciszewski, data from use of
in veterans. 2020 absolutist words.
Bryan et al., 2018 NA Audio no Predict ST with Adamou et al., 130.00 medical no Use AI to predict
recordings arousal and affect 2019 records SB.
regulation. Aladağ et al., 785.00 Posts yes Build classifier to
Cohn et al., 2009 28.00 Vocal no Use vocal 2018 identify posts.
expressions expression to Al-Mosaiwi & 1494.00 Posts yes Identify cognitive
detect ST. Johnstone, vulnerabilty as risk
Figueroa Saavedra NA Interview no Link acoustic 2018 factor.
et al., 2020 features to ST. Bantilan et al., 85,216.00 Messages no Build NLP
France et al., 2000 22.00 Audio no Investigate male 2020 written to algorithm to detect
recordings STB speech therapists suicide content.
features. Braithwaite 27,000.00 Tweets yes Validate ML model
Gideon et al., 2019 4078.00 Phone calls no Use emotion from et al., 2016 for SMD STB
speech to predict prediction.
ST. Cheng et al., 349,374.00 Words and yes Test computerized
Hashim et al., 2012 18.00 Audio no Analyze timing 2017 phrases language analysis.
recordings patterns of STB- Cohen, 2016 98,501.00 words yes Linguistic analysis
associated speech. of suicide bombers'
Hashim et al., 2017 NA Interview no Predict ST using notes.
acoustic features. Cohen et al., 249.00 Therapy no Test ML for voice
Keskinpala et al., NA Interview no Investigate STB- 2020 sessions analysis of therapy
2007 associated speech sessions.
features. Coppersmith 40,000,000.00 Tweets no Analyze lanuage
Landau et al., 2007 3.00 Interview, no Investigate STB- et al., 2016 and emoticons in
read & free associated speech posts.
speech features. Coppersmith 395,230.00 Posts no Use NLP and ML to
Ozdas et al., 2001 20.00 Audio no Analyze vocal et al., 2018 detect SB on social
recordings tract features. media.
Ozdas et al., 2004 NA Audio yes Investigate STB- Dieltjens et al., 72,874.00 Words no Predict end-of-life
recordings associated speech 2014 decision with
features. modal verbs.
Pestian et al., 2016 NA NA no Identify STB with Fahey et al., 2,889,190.00 Posts no Examine dirnual
ML. 2020 patterns of STB-
Pestian et al., 2017 371.00 Audio no Use ML on verbal associated texts.
recordings and nonverbal ST Fernandes 500.00 Documents no Develop NLP tools
cues. et al., 2018 to detect STB.
Scherer et al., 2013 16.00 Interview no Investigate STB- Glenn et al., 189,478.00 Text yes Use text messages
associated speech 2020 messages for suicide risk
features. detection.
Stasak et al., 2021 246.00 Audio no Use voice analysis Grant et al., 131,728.00 Posts no Extract informal
recordings to predict STBs. 2018 latent topics
Yingthawornsuk 2.00 Speech no Classify STB- related to ST.
et al., 2006 samples related vocal Grundlingh, 99.00 Suicide notes no Compare
features. 2018 fabricated and real
Yingthawornsuk 2.00 Speech no Classify STB- suicide notes.
et al., 2007 samples related acoustic Homan et al., 2000.00 Tweets yes Use SVM to classify
features. 2014 degree of distress
Yingthawornsuk & 2.00 Speech no Use acoustic in tweets.
Shiavi, 2008 samples features to Howard et al., 1588.00 Posts yes Use ML to identify
identify at-risk 2020 suicide risk posts.
pat. Ji et al., 2018 4143.00 Posts yes Identify ST on
Zhang et al., 2020 390.00 Audio no Predict STB with online platforms.
recordings voice features. Lester & 679.00 Suicide notes yes Identify gender
Leenaars, differences in
Note. LIWC, Linguistic Inquiry and Word Count software; ML, machine learning; 2016 suicide notes.
NA, not available; ST, suicidal thoughts; STB, suicidal thoughts and behaviors. Levis et al., 10,244.00 notes yes Use NLP on
2021 therapy notes.
Litvinova et al., 398,045.00 Words yes Classify texts based
Bingol (2018) used ML on over 10,000 forum posts that were catego­
2017 on texts
rized by a clinically trained rater for content associated with suicidal parameters.
thoughts. The text mining methods were useful and efficient in dis­ Liu et al., 2019 27,007.00 Comments yes Test Proactive
tinguishing posts from the subreddit SuicideWatch compared to other Suicide Prevention
non-suicide content subreddits. Suicidal thoughts were not assessed in Online approach.
Lv et al., 2015 4653.00 Posts yes Build and validate
the forum users, but the subreddit was specifically chosen to contain
a Chinese STB
suicide-related thoughts in the posts. Using lexical indices derived for dictionary.
example from the Linguistic Inquiry and Word Count (LIWC) text Nguyen et al., 5000.00 Posts yes Distinguish STB-
analysis program, suicidal thoughts were predicted with high accuracy. 2017 related from other
The authors do not reveal which specific indices contributed to this MH forums.
O'Dea et al., 2000.00 Tweets no
result. Interestingly, although posts with suicidal thought content were 2015
correctly discriminated from posts without, the models performed worse
(continued on next page)
in distinguishing non-suicidal depressive- or anxiety-content posts from

9
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

Table 8 (continued ) offer a specialized crisis intervention program. The clinician-rated social
Study # Datapoints Datapoint LIWC Aim media blog posts were analyzed with the Simplified Chinese LIWC tool
type prior to and after the crisis intervention. After the intervention
Use SVM to classify
compared with one month before, the use of death-related words (e.g.,
STB-related ‘die’ and ‘suicide’) decreased, while the use of future-related words
tweets. (‘after’ and ‘soon’) increased. The authors argue that this might be due
Ren et al., 2015 907.00 Posts no Examined to a decrease in suicidal thoughts and a willingness in seeking help, a
emotional traits in
relationship that was not statistically assessed. O'Dea et al. (2015)
suicide posts.
Sawhney et al., 500.00 Social media no Use adversarial gathered 2000 tweets containing phrases like ‘kill myself’ and ‘tired of
2021 useres learning to predict living’ and coded them with the help of a clinically trained rater as
suicidal risk. “strongly concerning” (e.g., ‘I might just kill myself’), “possibly con­
Song et al., NA NA no Use data mining on cerning” (i.e., default category for tweets with suicide-related content),
2016 social media for ST
prediction.
or “safe to ignore” (i.e., no evidence that the tweet suggested risk for
Tadesse et al., 7201.00 Posts no Automatic suicide). Of the tweets, 14% were coded as “strongly concerning” sug­
2020 recognition of STB- gesting that 32 tweets per day reflected a concerning level of STBs.
associated posts. Vioules, Moulahi, Azé, and Bringay (2018) used 5446 tweets (2381 from
Vioules et al., NA Tweets no Identify changes in
users with STBs, 3065 from users without) to quantify suicide-warning
2018 user's online
behavior. signs and to detect posts containing suicide-related content. The posts
Xu et al., 2021 5682.00 Therapy no Use NLP for suicide were labeled by raters who were not clinically trained and a mental
transcript risk prediction. health professional into four classes: “no distress” (e.g., everyday events
Zhong et al., 682.00 Pregnant no Screening for SB like work, leisure time, weekend activities), “minimal distress,” “mod­
2018 women with NLP.
erate distress,” and “severe distress” (e.g., mentions of self-harm, sui­
Note. AI, artificial intelligence; LIWC, Linguistic Inquiry and Word Count soft­ cidal thoughts, feelings of worthlessness, self-hate). Behavioral and
ware; MH, mental health; ML, machine learning; NA, not available; NLP, natural linguistic features were extracted using NLP, effectively capturing
language processing; SB, suicidal behavior; SMD, social media data; ST, suicidal warning signs in the posts and highlighting changes in online behavior.
thoughts; STB, suicidal thoughts and behaviors; SVM, support-vector machine. Among the most predictive linguistic features were greater use of first-
person pronouns, swear words, and intensifiers (e.g., ‘very,’
those with suicidal thought content. Al-Mosaiwi and Johnstone (2018) ‘extremely’).
used the LIWC tool to examine absolutist thinking, a cognitive bias in
anxiety and depression. Comparing individuals with suicidal thoughts to 3.3.2. Suicidal behavior
individuals with a diagnosis of both anxiety and depression and in­
dividuals without, the percentage of absolutist words of individuals with 3.3.2.1. Medical notes. Adamou et al., 2019 investigated lexical fea­
suicidal thoughts was significantly greater than in both other groups. tures of 130 free-text medical notes of psychiatric patients who died by
Importantly, the authors specifically searched a forum where users post suicide. Texts were segmented into single words focusing on word fre­
about their suicidal thoughts but did not assess the forum users' suicidal quency rather than on sentence structure. Using ML, the combination of
thought severity. Grant et al. (2018) extracted informal latent recurring linguistic features with information from structured data (e.g., de­
topics associated with suicidal thoughts from the Reddit platform Sui­ mographics, referrals, appointments, comprehensive assessments)
cideWatch. Using a neural network approach, which predicts neigh­ resulted in the best prediction of suicidal behaviors. Detailed informa­
boring words in a text based on vector representations of words, the tion on the linguistic features contributing to this result was, however,
identified clusters (e.g., ‘past,’ ‘suicidal,’ ‘havent,’ ‘attempted suicide,’ not presented. Levis, Westgate, Gui, Watts, and Shiner (2021) investi­
‘almost killed,’ and ‘failed attempt’) were matched successfully to the gated the predictive value of therapy notes about post-traumatic stress
twelve most common risk factors for suicide (e.g., “suicide attempt”). disorder patients and found associations between STB and need assess­
The posts were, however, not validated by a clinically trained rater in ments according to Maslow's hierarchy of needs. Individuals who died
terms of STB outcomes. Similarly, Tadesse, Lin, Xu, and Yang (2020) within three months of being diagnosed were more concerned with basic
investigated posts on Reddit with NLP and compared those with suicidal needs (e.g., food); those who died later were concerned with safety and
thought content to those without, albeit content was not assessed by conduct, emotional arousal and social affiliation; and those who died
clinically trained raters. Posts with suicidal thought content had more after a year were concerned with personal fulfillment and achievement.
mentions of hopelessness and frustration (e.g., ‘tired living,’ ‘hate However, NLP only performed better than an existing suicide risk model
tired’), anxiety (e.g., ‘I'm afraid’), guilt (e.g., ‘I am sorry’), regret (e.g., (i.e., the ML-based suicide prediction model ‘Engagement and Coordi­
‘never again’), loneliness (e.g., ‘no friend’), greater use of first-person nation for Health – Veterans Enhanced Treatment,’ REACH VET) for the
pronoun (e.g., ‘I'm,’ ‘I've never’), more negation (e.g., ‘no one,’ group of patients who died after one year, the largest sample with the
‘anymore,’ ‘I've never’), and more question marks (e.g., ‘Why is mankind greatest amount of data. Zhong et al. (2018) investigated the medical
afraid of death?’). records of pregnant women with a diagnosis that suggested previous
suicidal behavior, and a reference group of pregnant women without
3.3.1.3. Other social media data. Fahey, Boo, and Ueda (2020) exam­ such a diagnosis. NLP was used to classify the women as “patient reports
ined a comprehensive data set of tweets and suicide deaths mentions in feeling suicidal” and “suicidal behavior: none” based on an expert-
Japan to identify diurnal changes in the usage of the keyword “kietai” (“I defined list of terms associated with suicidal behavior. NLP was supe­
want to disappear”). The clinician-rated tweets showed a clear diurnal rior to diagnostic codes alone, with which over two-thirds of women
pattern such that the suicide-related keyword was mentioned most with potentially suicidal behavior and nearly half of those with
frequently between 1 am and 5 am in the younger cohort (15 to 44 years confirmed suicidal behavior would have been missed. Yet, the NLP
of age). Interestingly, this relationship was inversely correlated for the approach might have a high false-positive rate since the probability of a
cohort aged 45 and above. Night-time tweets also showed a different case with STBs was 30% compared to 78% who had been diagnosed with
network architecture and had more words related to self-disgust and the previous suicidal behavior.
suicidal thoughts than day-time tweets. Liu et al. (2019) used Proactive
Suicide Prevention Online, a new approach based on social media that 3.3.2.2. Social media. Cheng, Li, Kwok, Zhu, and Yip (2017) used ML on
aims to proactively identify individuals at high-risk for suicide and then

10
S. Homan et al.
data by social media users who were assessed by a non-clinically trained
rater using the Suicide Probability Scale. High suicide probability was
distinguished from severe anxiety among individuals who had written
about STBs. Those posts had more pronouns, less verbs, and greater
word counts. The number of work-related words was inversely related to
posts with suicidal behavior, depression, or anxiety related content.
Coppersmith, Ngo, Leary, and Wood (2016) used an automated labeling
approach on Twitter hashtags to capture suicide-related emotions. There
was a marked increase in “sadness” tweets a few weeks prior to a suicide
attempt, followed by a noticeable increase in “anger” and “sadness”
tweets following the week. Moreover, prior to a suicide attempt in­
dividuals started using more self-referential language. Subsequently,
Coppersmith, Leary, Crutchley, and Fine (2018) used public self-
reported data and data donated through OurDataHelps.org, a platform
where people voluntarily donate personal data including information
about past suicide attempts and social media usage. Using 395,230 so­
cial media posts, the authors applied NLP and deep learning to detect
quantifiable signals of suicide attempts in the posts that had been
assessed by a clinically trained rater. Based on lexical features, users
with a history of suicidal behavior could be distinguished from users
without; however, the authors did not indicate the specific lexical fea­
tures contributing to this distinction. Meanwhile, Dieltjens, Heynder­
ickx, Dees, and Vissers (2014) quantitatively and qualitatively analyzed
interviews with patients who requested euthanasia or physician-assisted
suicide. The most frequently used modal verb by patients was ‘can,’
besides ‘must’ and ‘may.’ The authors argued that such patients often
suffer unbearably and are unable to change their situations; thus,
euthanasia or physician-assisted suicide offers a way to act. Homan et al.
(2014) used the single LIWC dimension “sad” and suicide-specific search
terms to identify 2000 tweets, which were annotated by novices and
clinician experts as “happy,” “no distress,” “low distress,” or “high
distress.” Level of distress served as a proxy for suicidal behavior. The
ML algorithm achieved the best results for the expert-rated sample,
which might have been more sensitive to distress-cues, and distin­
guished “distress” from “non-distress” tweets with an accuracy of 64%.
3.3.2.3. Suicide notes. Cohen (2016) examined farewell notes of Pales­
tinian suicide bombers. The notes were translated into English and
segmented with the LIWC tool to identify common themes. They con­
tained a large number of words indicating explication and persuasion,
social concerns, especially positive emotions, and religious statements.
Further, word frequency analysis showed many extremizers and super­
latives (e.g., ‘any,’ ‘never,’ ‘uniquely,’ ‘excluding’). The most common
themes were not related to revenge but to ‘paradise,’ the bomber's
parents, and the wish to intercede on the family's behalf. Lester and
Leenaars (2016) compared suicide notes for gender differences in lexical
features. Notes written by women (N = 166) had more negations, more
words related to cognitive mechanisms, more present tense verbs, and
more indications of hopelessness and defeat.
3.3.2.4. Text messages. In a pilot study, Glenn, Nobles, Barnes, and
Teachman (2020) showed the potential of language analysis to retro­
spectively identify patterns in text messages. Before a suicide attempt,
text messages contained more words expressing anger and less
expressing positive emotions. However, the authors did not provide
examples of the specific word categories.
3.3.3. STBs
3.3.3.1. Forum posts. Howard et al. (2020) used lexicon-based tools (e.
g., LIWC, Valence Aware Dictionary and sEntiment Reasoner) on posts
from the Reachout.com forum with automated ML methods. Posts were
evaluated by a clinically trained rater. Suicide risk was predicted by
more negations (e.g., ‘not,’ ‘can't’), expressions of hopelessness (i.e., ‘no
hope left’), unwillingness or inability to continue (i.e., ‘I'm done,’ ‘I am
11
Clinical Psychology Review 95 (2022) 102161
too tired to’), and loneliness (i.e., ‘being isolated’). Nguyen et al. (2017)
adopted a similar approach with ML and forum posts. It was no indicated
whether the posts' content was evaluated by a clinically trained rater.
Suicide-related posts contained significantly more words with more than
six letters, more first-person singular pronouns, less second-person
pronouns, less third-person pronouns, less articles, more auxiliary
verbs, more present tense, more conjunctions, less negations, more
swear words, less words related to family, more words related to friends,
more words related to humans, more words indicating anxiety, more
tentative words, less words indicating certainty, less words indicating
inhibition, less words related to religion, and more words related to
death.
3.3.3.2. Suicide notes. Grundlingh (2018) examined authentic versus
fabricated suicide notes utilizing the appraisal theory to categorize
affect into more than two categories (i.e., “positive” and “negative”). Of
the 14 affect subcategories, only “trust” had enough power to discrim­
inate authentic and fabricated suicide notes. It was unclear whether the
notes were evaluated by a clinically trained rater.
3.3.3.3. Other social media data. Using the LIWC tool on Twitter data,
Braithwaite, Giraud-Carrier, West, Barnes, and Hanson (2016) found
that achievement- and religion-related words were associated with less
STBs, which they interpreted as a protective factor. The twitter users had
completed the Depressive Symptom Inventory (Suicide Subscale); it is
not clear whether this questionnaire was administered by a clinician. Lv,
Li, Liu, and Zhu (2015) created a Chinese suicide dictionary by selecting
words from social media posts (e.g., ‘dead end’ and ‘despair’) with the
help of a clinically trained rater. The dictionary was tested on the social
media platform Weibo to detect posts with suicide-related content
among posts without. The support vector machine (SVM) results showed
that dictionary-based identification enabled the detection of STB ex­
pressions in posts to judge suicide risk, comparable in accuracy to expert
ratings. Ren, Kang, and Quan (2015) compared blog posts of individuals
who had attempted suicide with randomly chosen posts of individuals
with suicidal thoughts. It is unknown whether the posts were assessed by
a clinically trained rater. The authors evaluated suicide-related emo­
tions and topics using a complex emotion topic model that assigned
words to emotion categories (“joy,” “love,” “expectation,” “anxiety,”
“sorrow,” “anger,” “hate,” and “surprise”) and intensities (scale from 1
“low” to 5 “strong”). For example, a blog post such as ‘Life was bitter, but
I was glad to have you around.’ would be classified in the emotion
categories “joy” and “love” with an intensity of 2 (low-medium). Posts of
individuals who died by suicide contained significantly more negative
emotion and language. Sawhney, Joshi, Gandhi, Jin, and Shah (2021)
used deep learning on Reddit posts. Suicide risk was assessed with the
CSSRS adapted for social media platforms. Posts were labeled by trained
clinicians using a pre-defined set of suicide related lexicons; however,
the content was not made clear. Deep learning outperformed other ap­
proaches possibly because of its ability to better learn representations of
users' mental state over time.
3.3.3.4. Therapy notes. Bantilan, Malgaroli, Ray, and Hull (2020)
designed an NLP algorithm to detect suicide-associated content in pa­
tients' written communication to their therapists (e.g., keywords like
‘suicide’ and numbers like ‘911’). Based on the 85,216 sentences from
1864 patients evaluated by a clinically trained rater, the algorithm
differentiated suicide-associated content from non-suicide-associated
content with good accuracy. Cohen et al. (2020) used NLP on speech
samples collected during therapy. The use of words (e.g., ‘and I,’
‘anymore,’ and ‘because of’) and the frequency of n-grams (i.e.,
contiguous sequence of n number of words) were relevant predictors of
suicide risk in adolescents. Using the same novel approach as Bantilan
et al. (2020), Xu et al. (2021) investigated 682 Cantonese therapy
transcripts between help-seekers and counselors that indicated an acute
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

crisis with STBs and were annotated by a clinically trained rater. The that the communication style on Twitter was more direct (e.g., ‘want to
novel approach incorporating expert knowledge on STBs enhanced the kill,’ ‘going to kill’), while on Reddit users shared more context infor­
performance of the text mining algorithm used to analyze the therapy mation. Suicidal thoughts-related posts contained more personal pro­
transcripts. It is unclear whether certain linguistic features contributed nouns (e.g., ‘I want to end my life.’), more words expressing negative
more than others to the positive suicide risk prediction of the model. emotions (e.g., ‘I was drowning in guilt and depression for several years
after.’), and were written in present and future tense (e.g., ‘I'm eventu­
3.4. Lexicon and first-person singular ally going to kill myself.’). O'Dea et al. (2021) examined the mental
health of individuals writing a blog over the course of 36 weeks. Blog
Of the 34 studies, 15 (20%) investigated both lexical and first-person owners who consented to the study completed the self-reported Patient
singular features in relation to STBs (Table 9). Health Questionnaire (PHQ-9). Item 9 was used to judge the severity of
suicidal thoughts. The features that best predicted suicidal thoughts
3.4.1. Suicidal thoughts were word count, auxiliary verbs (e.g., ‘am,’ ‘will,’ ‘have’), first-person
singular pronouns, and negations (e.g., ‘no,’ ‘not,’ ‘never’).
3.4.1.1. Social media. Lekkas, Klein, and Jacobson (2021) aimed to
predict acute suicidal thoughts in German adolescents using ML on 3.4.1.2. Speech samples. Pestian et al. (2020) analyzed a clinical inter­
linguistic features derived from a clinician-led interview via Instagram view that was repeated after 30 days. The frequencies of uni-grams, bi-
messenger and user activity on Instagram (e.g., number of followers, or grams, and LIWC word categories were extracted from the transcribed
pictures posted). Linguistic features were derived using the German interviews and used in an SVM classifier. Health-related (e.g., ‘clinic,’
version of the LIWC tool and included total word count, first-person ‘flu,’ ‘pill’) and body-related words (e.g., ‘cheeks,’ ‘hands,’ ‘spit’)
pronouns, emotion-related words, negative emotion-related words (e. decreased, while tentative words (e.g., ‘maybe,’ ‘perhaps,’ ‘guess’)
g., ‘hatred,’ ‘sadness’), and words representing cognitive processes (e.g., increased with decreasing suicidal thoughts. Overall, language charac­
‘think,’ ‘perhaps,’ and ‘else’). Only negative emotion-related words in teristics did not change significantly over time, making it possible to
the interviews predicted suicidal thoughts. Ji, Yu, Fung, Pan, and Long identify suicidal thoughts 30 days after the initial interview.
(2018) analyzed words, language, and topics of suicidal thought-related
posts (e.g., ‘I want to end my life tonight.’) compared with neutral posts 3.4.1.3. Other text sources. Brancu, Jobes, Wagner, Greene, and Fratto
(e.g., ‘I love this TV show and watch every week.’) from Twitter and (2016) analyzed the self- versus relational-oriented qualitative written
Reddit (i.e., SuicideWatch and other subreddits). Considering the answers of individuals receiving outpatient treatment who were expe­
different communication styles on Reddit (i.e., completely anonymous, riencing suicidal thoughts to the Suicide Status Form. The results with
not limited in character length) and Twitter (i.e., less anonymous as the LIWC showed that patients using self-oriented word (i.e., ‘I')
users might know each other, limited to 140 words), the authors found improved slower in therapy than patients whose responses were more
relational (included in the “family and friends” category). Brown et al.
Table 9 (2019) also used the LIWC tool with spoken and written language from
Liguistic feature descriptives of the first-person singular & lexicon-focused an interview and Instagram picture captions. Individuals with acute
studies. suicidal thoughts used significantly more negative emotion words and
Study # Datapoint LIWC Aim words expressing overall affect (e.g., ‘cried’) in interviews. However,
Datapoints type activity and language use on Instagram did not predict acute STBs.
Brancu et al., NA Posts yes Predict ST over the Further, Cook et al. (2016) investigated Spanish patients after discharge
2016 course of therapy. from a psychiatric hospital or emergency room after self-harm. Partici­
Brown et al., NA Instagram yes Predict ST from pants received an intervention in the form of repeated text messages
2019 captions Instagram posts. over multiple months and answered questions such as ‘Have you felt that
De Choudhury 46,969.00 Comments yes Develop logistic
you do not have the will to live?’ The free-text answers were analyzed
et al., 2016 regression classifier.
Cook et al., NA Questions no Use NLP to predict ST. using an NLP-based algorithm. The top 10 words/utterances associated
2016 with suicidal thoughts were ‘I told,’ ‘monotony,’ ‘Ritalin,’ ‘harassed,’
Kim et al., 56.00 Suicide yes Used linguistic and ‘we work,’ ‘race,’ ‘restrooms,’ ‘congested/sick,’ ‘I pronounce,’ and
2019 notes psychological features.
‘rejects.’
Leavitt et al., 114.00 Suicide yes Lexical analysis for SB
2021 notes, posts prediction.
Lekkas et al., 52.00 Social media yes Use ML to predict ST in 3.4.2. Suicidal behavior
2021 useres the past month.
Li et al., 2021 9611.00 Posts no New model for suicide
3.4.2.1. Forum posts. Litvinova, Seredin, Litvinova, and Romanchenko
risk assessment.
Nobles et al., 136,347.00 Text yes Predict STB from text
(2017) used mathematical models and the Russian version of the LIWC
2018 messages messages. tool to classify suicide- and non-suicide-related texts. Suicide-related
O'Dea et al., 2000.00 Tweets yes Define linguistic texts had more function words (i.e., ‘it,’ ‘was,’ ‘a,’ and ‘and’), verbs,
2017 profiles of tweets. conjunctions, words describing cognition, and inclusion words (i.e.,
O'Dea et al., 38.00 Social media yes Predict ST with social
‘and,’ ‘with,’ and ‘include’), fewer prepositions, and more pronouns.
2021 users media data.
Pestian et al., 66.00 Notes no Use ML to classify Interestingly, compared to English studies the authors found no increase
2008 suicide notes. in first-person pronouns in suicide-related texts. The known self-
Pestian et al., 2.00 Interview yes Use ML for STB- centered state of individuals with suicidal behavior might make them
2020 associated language
less receptive to the outer world without necessarily using more self-
change.
Sierra et al., 582.00 Posts yes Investigate linguistic
referential language (first-person pronouns).
2021 features realted to
suicide risk. 3.4.2.2. Suicide notes. Kim, Choi, Lee, and Sea (2019) also used suicide
Zhang et al., 3090.00 Notes yes Use ML to detect
notes and diaries with the Korean version of the LIWC tool. Suicide notes
2021 suicide notes online.
had fewer modifiers, numerals, and affixes/suffixes, more first- and
Note. LIWC, Linguistic Inquiry and Word Count software; ML, machine learning; second-person pronouns, fewer future tense verbs, and more negative
NA, not available; NLP, natural language processing; SB, suicidal behaviors; ST, words. Seeking to identify cues from written text regarding an increase
suicidal thoughts; STB, suicidal thoughts and behaviors.

12
S. Homan et al. Clinical Psychology Review 95 (2022) 102161

in suicide risk, Pestian et al. (2008) assessed linguistic features in Table 10

authentic and simulated suicide notes. Significant differences between Liguistic feature descriptives of the (morpho)syntax studies.
the notes related to linguistic and emotional characteristics, including Study # Datapoint LIWC Aim
word count, verbs, nouns, and prepositions. Datapoints type

Desmet & Hoste, 300,000.00 Posts no Test linguistic

3.4.3. STBs 2018 features for STB
prediction.
Fernández- 23.00 notes yes Assess linguistic
3.4.3.1. Suicide notes. Leavitt, Hong, and Walker (2021) examined
Cabana et al., features of suicide
suicide notes of persons who had died by suicide, and blogs concerned 2015 notes.
with “Depression,” “Suicide,” and, as a control, “Food.” First-person Lien & Zhang, 183,993.00 Survey data no Assess relation
singular pronouns were used in suicide notes more frequently than in 2020 between language
food-related posts, but less frequently than depression-related posts. and SB.
Teixeira et al., 139.00 Suicide no Analyze semantic
Negative emotion words were used in suicide notes less frequently than 2021 notes features of suicide
depression-related posts but more frequently than in food-related posts, notes.
with no significant difference for positive emotion words. Death-related
Note. LIWC, Linguistic Inquiry and Word Count software; SB, suicidal behaviors;
words were used more frequently in suicide notes than in food-related
STBs, suicidal thoughts and behaviors.
posts but not more frequently than in depression-related posts.
Religion-related words were used more frequently in suicide notes than
based on criteria commonly used for suicide risk assessment by the
in the blog posts. Zhang, Schoene, and Ananiadou (2021) investigated
Flemish Suicide Prevention Center. Using 300,000 social media posts,
suicide notes, final prison statements, and posts without STB content
lexical features, including token unigrams (e.g., ‘be’) and bigrams (e.g.,
using LIWC. Suicide notes had more personal pronouns and positive-
‘to be’), salient suicide-related terms, and negative or lack of positive
emotion words than posts but less than last statements, more adverbs,
polarity words, effectively predicted messages with suicidal thought-
negative-emotion words and death-references than both the other texts,
associated content. The authors did not provide examples of the lexi­
and fewer social-, family- and religion-related words than the other
cal feature categories.
texts.

3.5.1.2. Suicide notes. Teixeira, Talaga, Swanson, and Stella (2021)

3.4.3.2. Text messages. To assess the imminent risk of suicide among
examined the emotional and semantic structure of 139 suicide notes for
young adults, Nobles, Glenn, Kowsari, Teachman, and Barnes (2018)
linguistic predictors of suicidal thoughts and compared them to free-
screened psychology undergraduate students using a survey (e.g., ‘Have
association texts using network modeling. Suicide notes were con­
you ever made a suicide attempt?’ ‘Have you ever had thoughts of
structed to assemble pleasant concepts to avoid conflicting cognitions
wanting to kill yourself?’). The authors analyzed text messages of stu­
and were rich in positive perceptions related to “love,” “take,” “go,” and
dents with a history of STBs and found that the LIWC variables had weak
“way” while also conveying sadness and anticipation of the future. Yet,
discriminatory power. They attributed this to the informal language and
negative semantic associations were dominant regarding self-perception
shorthand often used in text messages, which might contain more
and perception of others. The authors inferred cognitive dissonance in
valuable information than, for example, forum posts, considering their
the mindsets of the suicide note authors.
intimate one-on-one setting.

3.5.2. Suicidal behavior

3.4.3.3. Social media. Li et al. (2021) used neural networks on data
from the social media platform SuicideWatch to improve suicide risk
3.5.2.1. Social media. Lien and Zhang (2020) investigated language
prediction. The dataset was assessed by a clinically trained rater for
differences in the use of future time references (FTR) (i.e., languages in
suicide-related content. Adding lexical features to the model signifi­
which future tense is common) as predictors for suicidal behaviors. For
cantly improved its performance in classifying posts regarding their
example, English is considered a “high” future time reference language,
suicide risk severity, but there was no discussion of which lexical fea­
as in the example ‘I will go to college next year,’ while Chinese is
tures contributed to this improvement. O'Dea, Larsen, Batterham,
considered “low” since the time reference ‘will’ is omitted. Based on data
Calear, and Christensen (2017) analyzed Twitter posts and found that
from the World Value Survey, the authors showed that countries with
suicide-related posts had more words, first-person pronouns, and death-
strong FTR languages (e.g., English, and French) have lower suicide
references than non-suicide-related posts. Sierra et al. (2021) collected
rates and individuals speaking this language have lower acceptance on
suicide- and depression-related data from two Facebook groups and
suicidal behaviors than countries and individuals with low FTR lan­
control data from Twitter, which they analyzed with the Spanish version
guages (e.g., Chinese, and German).
of the LIWC tool. Suicide-related posts contained similar word categories
to depression-related posts, but significantly different words compared
with the control group. Particularly, suicide-related posts contained 3.5.2.2. Suicide notes. Fernández-Cabana et al. (2015) assessed the
more negative emotion words conveyed anger, sadness, anxiety, and suicide notes of Spanish citizens who died by suicide as attested by a
death references. medical examiner. Notes written by women were longer, had more
words associated with affect, positive feelings and emotions, had more
past and future tense verbs, more spatial references, more negations,
3.5. Lexicon and (morpho)syntax more tentative words, more pronouns in first person plural, and more
punctuation marks.
Two studies investigated syntactic features of language as predictors
of STBs (Table 10). 3.6. Unspecified

3.5.1. Suicidal thoughts
3.5.1.1. Social media. Desmet and Hoste (2018) used a supervised text
classification to detect posts written in Dutch with suicidal thought
content (judged by a clinically trained rater). The authors included posts
Two studies investigated suicide-related features of language not
belonging to the categories of prosody, lexicon, (morpho)syntax, or first-
person singular (Table 11).

13
S. Homan et al.
Table 11
Liguistic feature descriptives of the unspecified studies.
Study # Datapoint LIWC Aim
Datapoints type
Audenaert NA Words no Investigate perfusion
et al., 2002 as suicidality marker.
Galatzer-Levy 20.00 Audio no Examine vocal cues for
et al., 2021 recordings ST.
Note. LIWC, Linguistic Inquiry and Word Count software; NA, not available; ST,
suicidal thoughts.
3.6.1. Suicidal thoughts
3.6.1.1. Audio recordings. Galatzer-Levy et al. (2021) investigated
audio recordings of patients experiencing suicidal thoughts who had
been assessed by the clinician-administered Beck Scale for Suicide
Ideation. Speech prevalence (i.e., the percentage of speech versus no
speech for the length of the recording) decreased with increasing sui­
cidal thought severity.
3.6.2. Suicidal behavior
3.6.2.1. Verbal fluency. Audenaert et al. (2002) used a verbal fluency
task on patients diagnosed with depression who had attempted suicide
and healthy controls. While performing the task (i.e., naming as many
words as possible starting with a certain letter within a predefined
timeframe) their cerebral blood flow was measured by single-photon
emission-computed tomography. Patients had a reduced ability to
perform the task and a blunted increase in blood flow in the prefrontal
cortex compared with controls. The poor verbal fluency might reflect
deficits in the prefrontal cortex, leading to an inability to mobilize
effective strategies in situations of competing choices and therefore
increasing the likelihood of suicide attempts.
4. Discussion
We systematically reviewed studies that assessed linguistic features
as predictors of STBs in clinical and non-clinical populations from the
last 20 years. We found that suicidal thoughts were predicted by more
intensifiers (Vioules et al., 2018) and superlatives (Belouali et al., 2021),
while suicidal behaviors were predicted by greater word count (Cheng
et al., 2017; Fernández-Cabana et al., 2015; Kim et al., 2019; Pestian
et al., 2008), greater usage of (first- and second-person) pronouns
(Cheng et al., 2017; Coppersmith et al., 2016; Kim et al., 2019; T. Zhang
et al., 2021), changes in the amount of verb usage (Cheng et al., 2017;
Fernández-Cabana et al., 2015; Kim et al., 2019; Litvinova et al., 2017;
Pestian et al., 2008), more prepend and multifunctional words (Cheng
et al., 2017), more nouns and prepositions (Pestian et al., 2008), and
fewer modifiers and numerals (Kim et al., 2019). Across the studies, we
found that most of the included studies focused on prosodic and lexical
features, while fewer focused on first-person singular pronouns and only
four focused on (morpho)syntactic features.
The studies investigated different linguistic features regarding the
specific phenomena (i.e., suicidal thoughts or behaviors), and findings
often overlapped. The overlap of findings reduces their predictive value
and specificity. For example, numerous studies investigated suicidal
thoughts and found increased use of first-person singular pronouns (Ji
et al., 2018; O'Dea et al., 2021; Tadesse et al., 2020; Vioules et al., 2018),
increased use of negative words including more death-references
(Brown et al., 2019; Desmet & Hoste, 2018; Fahey et al., 2020; Ji
et al., 2018; Lekkas et al., 2021; Liu et al., 2019; Tadesse et al., 2020;
Vioules et al., 2018), and more negations (O'Dea et al., 2021; Tadesse
et al., 2020). Several studies that investigated suicidal behavior also
found increased use of first-person pronouns (Cheng et al., 2017;
Coppersmith et al., 2016; Kim et al., 2019; T. Zhang et al., 2021),
14
Clinical Psychology Review 95 (2022) 102161
increased use of negative emotion words including death references
(Coppersmith et al., 2016; Glenn et al., 2020; Homan et al., 2014; Kim
et al., 2019; Lester & Leenaars, 2016; Litvinova et al., 2017; T. Zhang
et al., 2021), and more negations (Fernández-Cabana et al., 2015; Lester
& Leenaars, 2016). The few features that were exclusively predictive of
either suicidal thoughts or suicidal behaviors were only reported in a
limited number of studies. Perhaps complicating matters further, a
number of studies fell into the category of “STBs” and found the
following predictors: more jitter (i.e., variation in glottal pulse timing)
(Figueroa Saavedra et al., 2020; Ozdas et al., 2004), glottal flow indices
(e.g., greater slope or smaller indices) (Ozdas et al., 2004; S. Scherer
et al., 2013), bandwidth (Landau et al., 2007; Yingthawornsuk et al.,
2006), fundamental frequency F0 (Cohn et al., 2009; Figueroa Saavedra
et al., 2020; Pestian et al., 2017), spectrum-based measures (e.g., Mel
Frequency Cepstral Coeffiecients and Power Spectral Density) (France
et al., 2000; Hashim et al., 2017; Keskinpala et al., 2007; Landau et al.,
2007; Yingthawornsuk et al., 2006, 2007, 2008), and pause lengths
characterized by formants F1 and F3 (Figueroa Saavedra et al., 2020;
France et al., 2000; Pestian et al., 2017). This suggests that more
research is needed to compare individuals with suicidal thoughts and
behaviors with individuals who only have suicidal thoughts, to increase
the specificity of linguistic predictors. Another factor reducing speci­
ficity of the predictors is that the primary outcome was often unclear,
especially when the vague term ‘suicidality’ (Meyer et al., 2010) was
used. This term does not permit drawing conclusions about whether
suicidal thoughts or behaviors can be predicted (Klonsky et al., 2016), as
it combines suicidal thoughts, suicide attempts, suicides, and self-
injurious behaviors (Meyer et al., 2010). Therefore, we did not draw
conclusions about the findings of studies that did not specifically state
whether suicidal thoughts or suicidal behaviors were the primary
outcome.
Another matter of concern regarding the specificity of predictors is
the lack of differentiation between at-risk individuals with a history of
suicide attempts and at risk-individuals with suicidal thoughts but no
attempts (Klonsky et al., 2016). Suicidal thoughts often precede both
suicidal behavior suicide attempts, reflecting a progression from milder
to more severe stages in the suicidal process (Van Heeringen, Hawton, &
Williams, 2000). In addition, the studies varied in how STBs were
assessed. While some clinical studies administered clinical symptom-
specific scales (16, 21%), others relied on labeling posts “with suicidal
content” (9, 12%) or extracted posts from platforms specific to STBs (10,
13%). This stands in contrast to experimental studies such as random­
ized controlled trials (RCTs) where participants are meticulously
selected and assessed, groups are balanced, covariates are controlled for,
and where variables manipulated. There were 1 RCT (1%) (Cohn et al.,
2009), 1 cross-over trial (1%) (Nobles et al., 2018), and 7 controlled
trials (9%) (Levis et al., 2021; Ozdas et al., 2004; Pestian et al., 2016;
Pestian et al., 2017; Pestian et al., 2020; Scherer et al., 2013; Stasak
et al., 2021). There were 6 pilot studies (8%) (Adamou et al., 2019;
Brancu et al., 2016; J. Cohen et al., 2020; Dieltjens et al., 2014; Figueroa
Saavedra et al., 2020; Glenn et al., 2020) and a significant number of
observational studies (28, 37%) (Adam-Troian & Arciszewski, 2020; Al-
Mosaiwi & Johnstone, 2018; Audenaert et al., 2002; Braithwaite et al.,
2016; Brown et al., 2019; Bryan et al., 2018; Cook et al., 2016; De
Choudhury, Kiciman, Dredze, Coppersmith, & Kumar, 2016; Fernandes
et al., 2018; France et al., 2000; Hashim et al., 2012; Homan et al., 2014;
Keskinpala et al., 2007; Landau et al., 2007; Leavitt et al., 2021; Lekkas
et al., 2021; Li et al., 2021; O'Dea et al., 2015; O'Dea et al., 2017; Ozdas
et al., 2001; Pestian et al., 2008; Ren et al., 2015; Sawhney et al., 2021;
Sierra et al., 2021; Xu et al., 2021; Yingthawornsuk et al., 2006, 2007).
Therefore, some of the findings should be interpreted with caution.
Related to the specificity of predictors, the point in time at which STBs
are investigated is important. The month after discharge from an inpa­
tient psychiatric hospital stay is the period of highest risk for suicide
(Chung et al., 2017). Yet, the timepoint that STBs were assessed relative
to possible inpatient stays was unclear for most studies and thus the
S. Homan et al.
importance of the respective predictors should be considered with
caution.
Regarding data sources for written language, most studies focused on
social media data. There were three studies (Cook et al., 2016; Glenn
et al., 2020; Nobles et al., 2018) that relied on text message data, which
the authors claim contains valuable private information. Yet, this data
source is not generally accessible and thus not broadly available for use
in most prediction models. Regarding spoken language, the studies
varied in terms of the data source: 13 (17%) used free speech, 20 (27%)
used free response or interviews, 10 (13%) used read speech, and 4 (5%)
used vocal exercises. The data source and recorded data quality may
affect the analyzability of the extracted speech signal. When investi­
gating source features such as jitter, the use of vowels rather than
continuous speech is considered easier since suitable voiced sections can
be identified more easily (Laver, Hiller, & Beck, 1992). However,
Landau et al. (2007) used spontaneous and read speech from both sexes
and found that spontaneous speech could be modeled better than read
speech. This suggests that algorithms might use different strategies for
segmenting the speech signal to search for patterns.
Interestingly, studies on spoken language have focused mainly on
lexical or prosodic features, while (morpho)syntactic features of speech
have been largely neglected. Individual words will almost never appear
isolated, but rather within a string of words. Thus, investigating the
various syntactic rules and structures offers promising possibilities for
STB prediction. The studies that collected lexical and prosodic speech
features, mostly by counting certain lexical items, merely reveal what at-
risk individuals talk about. An in-depth syntactic analysis moves beyond
this information and may offer insights into how individuals express
themselves. Therefore, relying primarily on the LIWC tool (Pennebaker,
Francis, & Booth, 2001), as 30 (40%) studies in this review did, could be
significantly enhanced by coding and including syntactic features,
following the successful example of studies on psychosis (Corcoran et al.,
2018) and aging (Sung, Choi, Eom, Yoo, & Jeong, 2020). Additional
aspects in need for further analysis are culture and cultural differences.
Of the 75 studies included, 64 (85%) were conducted in Western culture
countries, with 41 (55%) from the United States alone. Especially
regarding language use, cultural differences should be considered when
assessing linguistic features as there may be differences not only in
grammar, sentence structure, and word choice but also in openness in
expressing thoughts and feelings (Granek, Nakash, Ariad, Shapira, &
Ben-David, 2020; Moleiro, 2018).
Many of the above-mentioned linguistic features, including word
count, self-referential language, and verb use, have been used in
regression or ML models to forecast STBs. Several studies demonstrated
that the severity of STBs can be extracted from social media posts with
ML methods (e.g., language-based classifiers) (Aladağ et al., 2018;
Cheng et al., 2017; Homan et al., 2014; Howard et al., 2020; Lekkas
et al., 2021; Nguyen et al., 2017). Posting frequency and language use
may thus offer insight into suicidal processes and allow for the timely
introduction of suicide prevention interventions. While social media
presents a unique opportunity to reach young people and for some in­
dividuals such interventions might mean rescue or support, the often-
unpredictable dynamics in the context of such intervention on social
media platforms may also contagiously affect others by spreading STBs
(Nobles et al., 2018). This might be particularly concerning in younger
age groups (Nobles et al., 2018). Therefore, sensitive interventions and
prevention on social media are required. In general, the initially
promising picture of acceptability and potential impact of a universal
suicide prevention campaign delivered through social media suggests
that it can be safe to utilize social media for the purpose of suicide
prevention (La Sala et al., 2021). This is important to consider when
developing ML methods to effectively use social media data to distin­
guish STB from non-STB content based on which prevention strategies
are being developed. In the special case of developing ML algorithms to
distinguish posts with non-suicidal depressive or anxiety-related content
from posts with STB content, a more contextual approach might be
15
Clinical Psychology Review 95 (2022) 102161
required considering the severity of STBs to prevent classifier failure
(Aladağ et al., 2018). Besides social media data, health records including
therapy notes (Bantilan et al., 2020; Cohen et al., 2020; Fernandes et al.,
2018; Levis et al., 2021; Xu et al., 2021), text message data (Cook et al.,
2016; Glenn et al., 2020; Nobles et al., 2018), suicide notes Pestian et al.
(2008), and voice samples (Belouali et al., 2021; Bryan et al., 2018;
Cohn et al., 2009; Figueroa Saavedra et al., 2020; France et al., 2000;
Gideon et al., 2019; Hashim et al., 2012, 2017; Keskinpala et al., 2007;
Landau et al., 2007; Ozdas et al., 2001; Ozdas et al., 2004; Pestian et al.,
2016; Pestian et al., 2017; Scherer et al., 2013; Stasak et al., 2021;
Yingthawornsuk et al., 2006, 2007) have proven useful in predicting
STBs.
With advances in modern (mobile) technology, ecological momen­
tary assessments can be exploited to collect real-time, real-world data on
individuals at risk, which is useful considering the fluctuating nature of
suicidal thoughts (Kleiman et al., 2019). Yet, some of these novel data
collection possibilities involve unique privacy and data security chal­
lenges (Roberts et al., 2021; Torous et al., 2021). While there are
established guidelines for risk management in online studies (Michaels,
Chu, Silva, Schulman, & Joiner, 2015), real-time assessment and
collection of mobile sensing data from individuals with STBs and at-risk
individuals may require different standards regarding, for example, the
necessary monitoring and recording of adverse events (for a review see
Huckvale, Nicholas, Torous, & Larsen, 2020; Kleiman et al., 2019;
Torous et al., 2021). Consequently, different organizations (e.g., Head to
Health in Australia and PsyberGuide in the US) have been formed to
ensure the safety and quality standard of mental health apps.
Some limitations merit comment. First, quantitatively assessing the
predictive power of linguistic indices based on the included studies in a
meta-analytic fashion is impossible considering the studies' heteroge­
neity in their methodological approaches, linguistic material, and,
crucially, STB outcomes. A few authoritative reviews do exist that
investigated similar aspects (Bernert et al., 2020; Cummins et al., 2015;
Demner-Fushman et al., 2009; Franklin et al., 2017; Kreimeyer et al.,
2017; Larsen et al., 2016; Notredame et al., 2019). Three focused on NLP
(Bernert et al., 2020; Demner-Fushman et al., 2009; Kreimeyer et al.,
2017), one on social media (Notredame et al., 2019), one on techno­
logical advances (Demner-Fushman et al., 2009), and one captured the
beginning of this newly evolving field of linguistic predictors for STBs
(Cummins et al., 2015). Thus, this review updates the previous findings
by systematically summarizing the body of literature in a comprehensive
way using descriptive statistics. Second, characterization of the samples
was difficult because basic demographics, such as age and sex, were
missing (32, 43%). In some studies, this was due to the study population
comprising an anonymous group of social media users. We used the
information available to categorize the study populations into age
groups (“adults” vs. “adolescents”). Third, the pre-defined categories of
“first-person singular pronoun” and “(morpho)syntax” had to be
adapted to include studies focusing on lexical features. This approach
was necessary because those features were investigated relation to lex­
ical features. Fourth, selection bias could potentially arise when per­
forming a literature search. To reduce such bias, we used a systematic
approach and adhered to the PRISMA guidelines while searching
through multiple databases (broad disciplinary as well as specialized
databases) (Rethlefsen et al., 2021). Fifth, there were several studies
where it was not clear, we categorized studies as “clinically trained
rater” if the first author had a psychology/psychiatry background. This
is only a proxy that should be considered with caution as this assumption
cannot be falsified. Finally, a considerable number of studies had a cross-
sectional design (22, 29%) (Bantilan et al., 2020; S. J. Cohen, 2016;
Coppersmith et al., 2018; Desmet & Hoste, 2018; Fernández-Cabana
et al., 2015; Gideon et al., 2019; Grant et al., 2018; Grundlingh, 2018;
Howard et al., 2020; Ji et al., 2018; Lien & Zhang, 2020; Litvinova et al.,
2017; Liu et al., 2019; Lv et al., 2015; Nguyen et al., 2017; Song, Song,
Seo, & Jin, 2016; Tadesse et al., 2020; Teixeira et al., 2021; L. Zhang,
Duvvuri, Chandra, Nguyen, & Ghomi, 2020; T. Zhang et al., 2021; Zhong
S. Homan et al.
et al., 2018; Vioules et al., 2018), while only three had a longitudinal
design (4%) (Belouali et al., 2021; Galatzer-Levy et al., 2021; O'Dea
et al., 2021). Study design is highly relevant both for suicide prevention
and the timely delivery of interventions. As suicidal thoughts fluctuate
within individuals even within a day (Kleiman & Nock, 2018), precise
estimation of this symptom together with co-occurring potential
markers are essential for reliable prediction. Therefore, future studies
are warranted that identify linguistic features in repeated measurement
designs in individuals with suicidal thoughts who attempted suicide, in
individuals with suicidal thoughts who never attempted suicide, and in
individuals without suicidal thoughts. Furthermore, the lack of
controlled studies calls into question the hypothesized linguistic fea­
tures, which in most cases were derived from observational retrospec­
tive studies. In many cases, this requires additional specification,
replication, and extension, while in other cases some of the predictors
are already standard in the field. Integrating these sources of predictive
features presents the greatest challenge but also the most promise in this
field.
In sum, there is a diverse field of research that investigates linguistic
features as predictors of STBs, with promising findings regarding pro­
sodic, lexical, first-person pronoun, and (morpho)syntactic features.
Considering the potential value of assessing additional linguistic fea­
tures, more studies focusing on (morpho)syntactic features as predictors
of STBs in at-risk clinical and non-clinical populations are needed. The
prosodic and lexical features that showed replicable specificity in pre­
dicting either suicidal thoughts or suicidal behaviors could be integrated
into daily clinical routines in the future. Such features could be captured
and analyzed during or near an actual clinical consultation or psycho­
therapy session when key assessments and clinical observations take
place, as well as in individuals at risk’ daily lives. This additional in­
formation may yield more precise STB evaluations with greater pre­
dictive power than based on patient- or clinician ratings alone, thus in
turn enriching clinical judgment. One possible future approach could be
to record patients' voices during therapy sessions, which is often done
for quality assurance and in the context of supervision and feed the in­
formation into trained algorithms and simultaneously analyzed. Based
on such an “online” feedback mechanism, the results could be imme­
diately integrated by the therapist who would then adjust the planned
procedure according to, for instance, a suggested STB risk detection
algorithm. Another possible future approach could be that patients fill in
electronic diaries, from which information on STBs can be extracted and
analyzed in real-time. Some of this data could be collected passively,
without much patient effort, from mobile sensors in the background
without active interaction on spoken and written language. If suicide
risk falls above a predetermined threshold, as indicated by the algo­
rithm, a cascade of just-in-time adaptive interventions could be trig­
gered (Künzler et al., 2019) or the treating clinician or support system
alerted (Larsen et al., 2016). The first scenario is one in which acoustic
features could be used as support for clinicians' judgment during ther­
apy. The second scenario is one in which acoustic features could be used
outside the therapist's office, namely between therapy sessions, to
monitor patients' mental health and changes in STBs, which could
immediately be fed back to the therapist, however, and allow for a more
comprehensive care, and to intervene potentially even by triggering a
just-in-time adaptive intervention. There are thus unparalleled possi­
bilities for data collection, including text mining, and accessible speech
and text databases, in- and outside the therapist's office.
Author contributions
S.H., B.K., E.S and G.S. developed the research question and
conceptualized the study. All authors agreed on the studies included. M.
G., N.K., M.D., S.M., and A.-M.B. screened the studies and extracted the
data. A.-M.M. and S.B. categorized the studies. A.-M.M., S.B. G.S., E.S.,
and A.M. provided their expertise on linguistic issues. S.H. wrote the
first draft. All authors contributed to and approved the final manuscript
16
Clinical Psychology Review 95 (2022) 102161
version.
Funding source
This research has been supported by the Swiss National Science
Foundation's Digital Lives Funding Scheme (#10DL12_183251) to Birgit
Kleim and Urte Scholz.
Disclosure statement
The authors have no conflicts of interest to disclose.
Data availability
I have shared the link to all data and code in the manuscript file.
Acknowledgements
The authors thank Ellen Ji, Ph.D., for proof-reading the manuscript
and her valuable input on language choice.
References
Adamou, M., Antoniou, G., Greasidou, E., Lagani, V., Charonyktakis, P., Tsamardinos, I.,
& Doyle, M. (2019). Toward automatic risk assessment to support suicide
prevention. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 40(4),
249–256.
Adam-Troian, J., & Arciszewski, T. (2020). Absolutist words from search volume data
predict state-level suicide rates in the United States. Clinical Psychological Science: A
Journal of the Association for Psychological Science, 8(4), 788–793.
Aladağ, A. E., Muderrisoglu, S., Akbas, N. B., Zahmacioglu, O., & Bingol, H. O. (2018).
Detecting suicidal ideation on forums: Proof-of-concept study. Journal of Medical
Internet Research, 20(6), Article e215.
Al-Mosaiwi, M., & Johnstone, T. (2018). In an absolute state: Elevated use of absolutist
words is a marker specific to anxiety, depression, and suicidal ideation. Clinical
Psychological Science: A Journal of the Association for Psychological Science, 6(4),
529–542.
Altmann, G. (1981). Zur funktionalanalyse in der linguistik (pp. 25–32). Tübingen: Narr:
Forms and Functions.
Audenaert, K., Goethals, I., Van Laere, K., Lahorte, P., Brans, B., Versijpt, J., Vervaet, M.,
Beelaert, L., Van Heeringen, K., & Dierckx, R. (2002). SPECT neuropsychological
activation procedure with the verbal fluency test in attempted suicide patients.
Nuclear Medicine Communications, 23(9), 907–916.
Balaban, H.Ö., & Hohenberger, A. (2020). The development of narrative skills in turkish-
speaking children: A complexity approach. PLoS One, 15(5), Article e0232579.
Bantilan, N., Malgaroli, M., Ray, B., & Hull, T. D. (2020). Just in time crisis response:
Suicide alert system for telemedicine psychotherapy settings. Psychotherapy Research,
1–11.
Barak-Corren, Y., Castro, V. M., Nock, M. K., Mandl, K. D., Madsen, E. M., Seiger, A., , …
Klann, J. G., et al. (2020). Validation of an electronic health record–based suicide
risk prediction modeling approach across multiple health care systems. JAMA
Network Open, 3(3), e201262.
Belouali, A., Gupta, S., Sourirajan, V., Yu, J., Allen, N., Alaoui, A., … Reinhard, M. J.
(2021). Acoustic and language analysis of speech for suicidal ideation among US
veterans. BioData Mining, 14(1), 1–17.
Bernert, R. A., Hilberg, A. M., Melia, R., Kim, J. P., Shah, N. H., & Abnousi, F. (2020).
Artificial intelligence and suicide prevention: A systematic review of machine
learning investigations. International Journal of Environmental Research and Public
Health, 17(16), 5929.
Braithwaite, S. R., Giraud-Carrier, C., West, J., Barnes, M. D., & Hanson, C. L. (2016).
Validating machine learning algorithms for twitter data against established measures
of suicidality. JMIR Mental Health, 3(2), Article e21.
Brancu, M., Jobes, D., Wagner, B. M., Greene, J. A., & Fratto, T. A. (2016). Are there
linguistic markers of suicidal writing that can predict the course of treatment? A
repeated measures longitudinal analysis. Archives of Suicide Research, 20(3),
438–450.
Brown, R. C., Bendig, E., Fischer, T., Goldwich, A. D., Baumeister, H., & Plener, P. L.
(2019). Can acute suicidality be predicted by instagram data? Results from
qualitative and quantitative language analyses. PLoS One, 14(9), Article e0220623.
Bryan, C. J., Baucom, B. R., Crenshaw, A. O., Imel, Z., Atkins, D. C., Clemans, T. A., …
Rudd, M. D. (2018). Associations of patient-rated emotional bond and vocally
encoded emotional arousal among clinicians and acutely suicidal military personnel.
Journal of Consulting and Clinical Psychology, 86(4), 372.
Cheng, Q., Li, T. M., Kwok, C.-L., Zhu, T., & Yip, P. S. (2017). Assessing suicide risk and
emotional distress in chinese social media: A text mining and machine learning
study. Journal of Medical Internet Research, 19(7), Article e243.
Choo, C. C., Kuek, J. H., & Burton, A. A. (2018). Smartphone applications for mindfulness
interventions with suicidality in asian older adults: A literature review. International
Journal of Environmental Research and Public Health, 15(12), 2810.
S. Homan et al.
Chu, C., Buchman-Schmitt, J. M., Stanley, I. H., Hom, M. A., Tucker, R. P., Hagan, C. R., ,
… Ringer, F. B., et al. (2017). The interpersonal theory of suicide: A systematic
review and meta-analysis of a decade of cross-national research. Psychological
Bulletin, 143(12), 1313.
Chung, D. T., Ryan, C. J., Hadzi-Pavlovic, D., Singh, S. P., Stanton, C., & Large, M. M.
(2017). Suicide rates after discharge from psychiatric facilities: A systematic review
and meta-analysis. JAMA Psychiatry, 74(7), 694–702.
Cohen, J., Wright-Berryman, J., Rohlfs, L., Wright, D., Campbell, M., Gingrich, D.,
Santel, D., & Pestian, J. (2020). A feasibility study using a machine learning suicide
risk prediction model based on open-ended interview language in adolescent therapy
sessions. International Journal of Environmental Research and Public Health, 17(21),
8187.
Cohen, S. J. (2016). Mapping the minds of suicide bombers using linguistic methods: The
corpus of palestinian suicide bombers’ farewell letters (CoPSBFL). Studies in Conflict
& Terrorism, 39(7–8), 749–780.
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., … De la
Torre, F. (2009). Detecting depression from facial actions and vocal prosody (pp. 1–7).
Cook, B. L., Progovac, A. M., Chen, P., Mullin, B., Hou, S., & Baca-Garcia, E. (2016).
Novel use of natural language processing (NLP) to predict suicidal ideation and
psychiatric symptoms in a text-based mental health intervention in Madrid. In
Computational and mathematical methods in medicine, 2016.
Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018). Natural language processing
of social media as screening for suicide risk. Biomedical Informatics Insights, 10,
1178222618792860.
Coppersmith, G., Ngo, K., Leary, R., & Wood, A. (2016). Exploratory analysis of social
media prior to a suicide attempt. In Proceedings of the third workshop on computational
linguistics and clinical psychology (pp. 106–117).
Corcoran, C. M., Carrillo, F., Fernández-Slezak, D., Bedi, G., Klim, C., Javitt, D. C., …
Cecchi, G. A. (2018). Prediction of psychosis across protocols and risk cohorts using
automated language analysis. World Psychiatry, 17(1), 67–75.
Crosby, A., Ortega, L., & Melanson, C. (2011). Self-directed violence surveillance; uniform
definitions and recommended data elements.
Culicover, P. W. (2014). Constructions, complexity, and word order variation. Measuring
Linguistic Complexity, 148–178.
Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015).
A review of depression and suicide risk assessment using speech analysis. Speech
Communication, 71, 10–49.
Davidson, C. L., Anestis, M. D., & Gutierrez, P. M. (2017). Ecological momentary
assessment is a neglected methodology in suicidology. Archives of Suicide Research,
21(1), 1–11.
De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., & Kumar, M. (2016).
Discovering shifts to suicidal ideation from mental health content in social media (pp.
2098–2110).
Delage, H., & Frauenfelder, U. H. (2019). Syntax and working memory in typically-
developing children: Focus on syntactic complexity. Language, Interaction and
Acquisition, 10(2), 141–176.
Demner-Fushman, D., Chapman, W. W., & McDonald, C. J. (2009). What can natural
language processing do for clinical decision support? Journal of Biomedical
Informatics, 42(5), 760–772.
Desmet, B., & Hoste, V. (2018). Online suicide prevention through optimised text
classification. Information Sciences, 439, 61–78.
Dieltjens, S. M., Heynderickx, P. C., Dees, M. K., & Vissers, K. C. (2014). Linguistic
analysis of face-to-face interviews with patients with an explicit request for
euthanasia, their closest relatives, and their attending physicians: The use of modal
verbs in dutch. Pain Practice, 14(4), 324–331.
Esmaeelpour, E., & Sasani, F. (2018). Linguistic complexity of mixed episode in sadegh
hedayat’s letters: The effect of bipolar mood disorder. In RAIS Conference
Proceedings-the 11th International RAIS Conference on Social Sciences.
Fahey, R. A., Boo, J., & Ueda, M. (2020). Covariance in diurnal patterns of suicide-related
expressions on twitter and recorded suicide deaths. Social Science & Medicine, 253,
Article 112960.
Fernandes, A. C., Dutta, R., Velupillai, S., Sanyal, J., Stewart, R., & Chandran, D. (2018).
Identifying suicide ideation and suicidal attempts in a psychiatric clinical research
database using natural language processing. Scientific Reports, 8(1), 1–10.
Fernández-Cabana, M., Jiménez-Féliz, J., Alves-Pérez, M. T., Mateos, R., Gómez-Reino
Rodríguez, I., & García-Caballero, A. (2015). Linguistic analysis of suicide notes in
Spain. The European Journal of Psychiatry, 29(2), 145–155.
Fernandez-Duque, D. (2009). Cognitive and neural underpinnings of syntactic
complexity. In Syntactic complexity: Diachrony, acquisition, neuro-cognition, evolution.
Amsterdam: John Benjamins Publishing Company.
Figueroa Saavedra, C., Otzen Hernández, T., Alarcón Godoy, C., Ríos Pérez, A., Frugone
Salinas, D., & Lagos Hernández, R. (2020). Association between suicidal ideation and
acoustic parameters of university students’ voice and speech: A pilot study.
Logopedics, Phoniatrics, Vocology, 1–8.
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000). Acoustical
properties of speech as indicators of depression and suicidal risk. IEEE Transactions
on Biomedical Engineering, 47(7), 829–837.
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., …
Nock, M. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis
of 50 years of research. Psychological Bulletin, 143(2), 187.
Galatzer-Levy, I., Abbas, A., Ries, A., Homan, S., Sels, L., Koesmahargyo, V., Yadav, V.,
Colla, M., Scheerer, H., Vetter, S., et al. (2021). Validation of visual and auditory
digital markers of suicidality in acutely suicidal psychiatric inpatients: Proof-of-
concept study. Journal of Medical Internet Research, 23(6), Article e25199.
Gideon, J., Schatten, H. T., McInnis, M. G., & Provost, E. M. (2019). Emotion recognition
from natural phone conversations in individuals with and without recent suicidal
17
Clinical Psychology Review 95 (2022) 102161
ideation. Proc. Interspeech, 2019, 3282–3286. https://doi.org/10.21437/
Interspeech.2019-1830
Givón, T. (1983). Topic continuity in discourse: An introduction. In , 3. Topic Continuity
in Discourse: A Quantitative Cross-Language Study (pp. 1–42).
Givón, T., & Shibatani, M. (2009). Syntactic complexity: Diachrony, acquisition, neuro-
cognition, evolution (Vol. 85). John Benjamins Publishing.
Glenn, J. J., Nobles, A. L., Barnes, L. E., & Teachman, B. A. (2020). Can text messages
identify suicide risk in real time? A within-subjects pilot examination of temporally
sensitive markers of suicide risk. Clinical Psychological Science: A Journal of the
Association for Psychological Science, 8(4), 704–722.
Granek, L., Nakash, O., Ariad, S., Shapira, S., & Ben-David, M. A. (2020). The role of
culture/ethnicity in communicating with cancer patients about mental health
distress and suicidality. Culture, Medicine and Psychiatry, 44(2), 214–229.
Grant, R. N., Kucher, D., León, A. M., Gemmell, J. F., Raicu, D. S., & Fodeh, S. J. (2018).
Automatic extraction of informal topics from online suicidal ideation. BMC
Bioinformatics, 19(8), 57–66.
Grundlingh, L. (2018). Exploring the possibility of using appraisal theory to determine
the legitimacy of suicide notes. Lingua, 214, 1–10.
Hashim, N. W., Wilkes, M., Salomon, R., & Meggs, J. (2012). Analysis of timing pattern of
speech as possible indicator for near-term suicidal risk and depression in male
patients. International Proceedings of Computer Science and Information Technology, 58,
6.
Hashim, N. W., Wilkes, M., Salomon, R., Meggs, J., & France, D. J. (2017). Evaluation of
voice acoustics as predictors of clinical depression scores. Journal of Voice, 31(2),
256–e1.
Homan, C., Johar, R., Liu, T., Lytle, M., Silenzio, V., & Alm, C. O. (2014). Toward macro-
insights for suicide prevention: Analyzing fine-grained distress at scale. In
Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From
Linguistic Signal to Clinical Reality (pp. 107–117).
Howard, D., Maslej, M. M., Lee, J., Ritchie, J., Woollard, G., & French, L. (2020). Transfer
learning for risk classification of social media posts: Model evaluation study. Journal
of Medical Internet Research, 22(5), Article e15371.
Huckvale, K., Nicholas, J., Torous, J., & Larsen, M. E. (2020). Smartphone apps for the
treatment of mental health conditions: Status and considerations. Current Opinion in
Psychology, 36, 65–70.
Ji, S., Yu, C. P., Fung, S., Pan, S., & Long, G. (2018). Supervised learning for suicidal
ideation detection in online user content. Complexity, 2018.
Joiner, T. E., Van Orden, K. A., Witte, T. K., & Rudd, M. D. (2009). The interpersonal theory
of suicide: Guidance for working with suicidal clients. American Psychological
Association.
Keskinpala, H. K., Yingthawornsuk, T., Wilkes, D. M., Shiavi, R. G., & Salomon, R. M.
(2007). Screening for high risk suicidal states using mel-cepstral coefficients and
energy in frequency bands. In 2007 15th European Signal Processing Conference (pp.
2229–2233).
Kim, K., Choi, S., Lee, J., & Sea, J. (2019). Differences in linguistic and psychological
characteristics between suicide notes and diaries. The Journal of General Psychology,
146(4), 391–416.
Kleiman, E. M., Glenn, C. R., & Liu, R. T. (2019). Real-time monitoring of suicide risk
among adolescents: Potential barriers, possible solutions, and future directions.
Journal of Clinical Child and Adolescent Psychology, 48(6), 934–946.
Kleiman, E. M., & Nock, M. K. (2018). Real-time assessment of suicidal thoughts and
behaviors. Current Opinion in Psychology, 22, 33–37.
Klonsky, E. D., May, A. M., & Saffer, B. Y. (2016). Suicide, suicide attempts, and suicidal
ideation. Annual Review of Clinical Psychology, 12, 307–330.
Kraepelin, E. (1921). Manic depressive insanity and paranoia. The Journal of Nervous and
Mental Disease, 53(4), 350.
Kreimeyer, K., Foster, M., Pandey, A., Arya, N., Halford, G., Jones, S. F., … Botsis, T.
(2017). Natural language processing systems for capturing and standardizing
unstructured clinical information: A systematic review. Journal of Biomedical
Informatics, 73, 14–29.
Kreuze, E., Jenkins, C., Gregoski, M., York, J., Mueller, M., Lamis, D. A., & Ruggiero, K. J.
(2017). Technology-enhanced suicide prevention interventions: A systematic review.
Journal of Telemedicine and Telecare, 23(6), 605–617.
Krifka, M. (2007). Approximate interpretation of number words.
Künzler, F., Mishra, V., Kramer, J.-N., Kotz, D., Fleisch, E., & Kowatsch, T. (2019).
Exploring the state-of-receptivity for mhealth interventions. Proceedings of the ACM
on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(4), 1–27.
La Sala, L., Teh, Z., Lamblin, M., Rajaram, G., Rice, S., Hill, N. T., … Robinson, J. (2021).
Can a social media intervention improve online communication about suicide? A
feasibility study examining the acceptability and potential impact of the# chatsafe
campaign. PLoS One, 16(6), Article e0253278.
Lai, M. H., Maniam, T., Chan, L. F., & Ravindran, A. V. (2014). Caught in the web: A
review of web-based suicide prevention. Journal of Medical Internet Research, 16(1),
Article e30.
Landau, M., Yingthawornsuk, T., Wilkes, D. M., Shiavi, R. G., & Salomon, R. M. (2007).
Predicting severity of mental state using vocal output characteristics. In Fifth
International Workshop on Models and Analysis of Vocal Emissions for Biomedical
Applications.
Larsen, J. L. S., Frandsen, H., & Erlangsen, A. (2016). MYPLAN–a mobile phone
application for supporting people at risk of suicide. Crisis: The Journal of Crisis
Intervention and Suicide Prevention, 37(3), 236–240.
Laver, J., Hiller, S., & Beck, J. M. (1992). Acoustic waveform perturbations and voice
disorders. Journal of Voice, 6(2), 115–126.
Law, K. C., Jin, H. M., & Anestis, M. D. (2018). The intensity of suicidal ideation at the
worst point and its association with suicide attempts. Psychiatry Research, 269,
524–528.
S. Homan et al.
Leavitt, J., Hong, J. H., & Walker, R. L. (2021). Paradoxical positivity: Suicide notes use
less distressed language than blogs about depression, suicidal thoughts, and even
cooking. Suicide & Life-Threatening Behavior, 51(5), 1005–1014.
Lekkas, D., Klein, R. J., & Jacobson, N. C. (2021). Predicting acute suicidal ideation on
instagram using ensemble machine learning models. Internet Interventions, 25, Article
100424.
Lester, D., & Leenaars, A. (2016). A comparison of suicide notes written by men and
women. Death Studies, 40(3), 201–203.
Levis, M., Westgate, C. L., Gui, J., Watts, B. V., & Shiner, B. (2021). Natural language
processing of clinical mental health notes may add predictive value to existing
suicide risk models. Psychological Medicine, 51(8), 1382–1391.
Li, J., Zhang, S., Zhang, Y., Lin, H., Wang, J., et al. (2021). Multifeature fusion attention
network for suicide risk assessment based on social media: Algorithm development
and validation. JMIR Medical Informatics, 9(7), Article e28227.
Lien, D., & Zhang, S. (2020). Words matter life: The effect of language on suicide
behavior. Journal of Behavioral and Experimental Economics, 86, Article 101536.
Limburg, K., Watson, H. J., Hagger, M. S., & Egan, S. J. (2017). The relationship between
perfectionism and psychopathology: A meta-analysis. Journal of Clinical Psychology,
73(10), 1301–1326.
Litvinova, T. A., Seredin, P. V., Litvinova, O. A., & Romanchenko, O. V. (2017).
Identification of suicidal tendencies of individuals based on the quantitative analysis
of their internet texts. Computación y Sistemas, 21(2), 243–252.
Liu, X., Liu, X., Sun, J., Yu, N. X., Sun, B., Li, Q., & Zhu, T. (2019). Proactive suicide
prevention online (PSPO): Machine identification and crisis management for chinese
social media users with suicidal thoughts and behaviors. Journal of Medical Internet
Research, 21(5), Article e11705.
Lv, M., Li, A., Liu, T., & Zhu, T. (2015). Creating a chinese suicide dictionary for
identifying suicide risk on social media. PeerJ, 3, Article e1455.
Madan, A., Frueh, B. C., Allen, J. G., Ellis, T. E., Rufino, K. A., Oldham, J. M., &
Fowler, J. C. (2016). Psychometric reevaluation of the columbia-suicide severity
rating scale: Findings from a prospective, inpatient cohort of severely mentally ill
adults. The Journal of Clinical Psychiatry, 77(7), e867–e873.
Mayes, T. L., Killian, M., Rush, A. J., Emslie, G. J., Carmody, T., Kennard, B. D., …
Trivedi, M. H. (2020). Predicting future suicidal events in adolescents using the
concise health risk tracking self-report (CHRT-SR). Journal of Psychiatric Research,
126, 19–25.
McHugh, C. M., Corderoy, A., Ryan, C. J., Hickie, I. B., & Large, M. M. (2019).
Association between suicidal ideation and suicide: Meta-analyses of odds ratios,
sensitivity, specificity and positive predictive value. BJPsych Open, 5(2).
Meehan, W., Mannix, R., Zafonte, R., & Pascual-Leone, A. (2015). Chronic traumatic
encephalopathy and athletes. Neurology, 85(17), 1504–1511.
Melia, R., Francis, K., Hickey, E., Bogue, J., Duggan, J., O’Sullivan, M., & Young, K.
(2020). Mobile health technology interventions for suicide prevention: Systematic
review. JMIR mHealth and uHealth, 8(1), Article e12516.
Meyer, R. E., Salzman, C., Youngstrom, E. A., Clayton, P. J., Goodwin, F. K., Mann, J. J., ,
… Greden, J. F., et al. (2010). Suicidality and risk of suicide-definition, drug safety
concerns, and a necessary target for drug development: A consensus statement. The
Journal of Clinical Psychiatry, 71(8), e1–e21.
Michaels, M. S., Chu, C., Silva, C., Schulman, B. E., & Joiner, T. (2015). Considerations
regarding online methods for suicide-related research and suicide risk assessment.
Suicide & Life-Threatening Behavior, 45(1), 10–17.
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., …
Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-
analysis protocols (PRISMA-p) 2015 statement. Systematic Reviews, 4(1), 1.
Moleiro, C. (2018). Culture and psychopathology: New perspectives on research,
practice, and clinical training in a globalized world. Frontiers in Psychiatry, 9, 366.
Musan, R. (2002). Informationsstrukturelle dimensionen im deutschen: Zur variation der
wortstellung im mittelfeld. Zeitschrift für Germanistische Linguistik, 30(2), 198.
Nguyen, T., O’Dea, B., Larsen, M., Phung, D., Venkatesh, S., & Christensen, H. (2017).
Using linguistic and topic analysis to classify sub-groups of online depression
communities. Multimedia Tools and Applications, 76(8), 10653–10676.
Nobles, A. L., Glenn, J. J., Kowsari, K., Teachman, B. A., & Barnes, L. E. (2018).
Identification of imminent suicide risk among young adults using text messages. In
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp.
1–11).
Nock, M. K., Prinstein, M. J., & Sterba, S. K. (2009). Revealing the form and function of
self-injurious thoughts and behaviors: A real-time ecological assessment study
among adolescents and young adults. Journal of Abnormal Psychology, 118(4), 816.
Notredame, C.-E., Morgiève, M., Morel, F., Berrouiguet, S., Azé, J., & Vaiva, G. (2019).
Distress, suicidality, and affective disorders at the time of social networks. Current
Psychiatry Reports, 21(10), 98.
O’Dea, B., Boonstra, T. W., Larsen, M. E., Nguyen, T., Venkatesh, S., & Christensen, H.
(2021). The relationship between linguistic expression in blog content and
symptoms of depression, anxiety, and suicidal thoughts: A longitudinal study. PLoS
One, 16(5), Article e0251787.
O’Dea, B., Larsen, M. E., Batterham, P. J., Calear, A. L., & Christensen, H. (2017).
A linguistic analysis of suicide-related twitter posts. Crisis: The Journal of Crisis
Intervention and Suicide Prevention, 38(5), 319.
O’Dea, B., Wan, S., Batterham, P. J., Calear, A. L., Paris, C., & Christensen, H. (2015).
Detecting suicidality on twitter. Internet Interventions, 2(2), 183–188.
Ozdas, A., Shiavi, R. G., Silverman, S. E., Silverman, M. K., & Wilkes, D. M. (2004).
Investigation of vocal jitter and glottal flow spectrum as possible cues for depression
and near-term suicidal risk. IEEE Transactions on Biomedical Engineering, 51(9),
1530–1540.
Ozdas, A., Shiavi, R. G., Wilkes, D. M., Silverman, M. K., & Silverman, S. E. (2001).
Analysis of vocal tract characteristics for near-term suicidal risk assessment. In
18
Clinical Psychology Review 95 (2022) 102161
Second International Workshop on Models and Analysis of Vocal Emissions for
Biomedical Applications.
Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word
count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, 71(2001), 2001.
Pestian, J., Grupp-Phelan, J., Bretonnel Cohen, K., Meyers, G., Richey, L. A.,
Matykiewicz, P., & Sorter, M. T. (2016). A controlled trial using natural language
processing to examine the language of suicidal adolescents in the emergency
department. Suicide & Life-Threatening Behavior, 46(2), 154–159.
Pestian, J., Matykiewicz, P., Grupp-Phelan, J., Lavanier, S. A., Combs, J., & Kowatch, R.
(2008). Using natural language processing to classify suicide notes. In Proceedings of
the Workshop on Current Trends in Biomedical Natural Language Processing (pp. 96–97).
Pestian, J., Santel, D., Sorter, M., Bayram, U., Connolly, B., Glauser, T., DelBello, M.,
Tamang, S., & Cohen, K. (2020). A machine learning approach to identifying changes
in suicidal language. Suicide & Life-Threatening Behavior, 50(5), 939–947.
Pestian, J., Sorter, M., Connolly, B., Bretonnel Cohen, K., McCullumsmith, C., Gee, J. T.,
… Group, S. R. (2017). A machine learning approach to identifying the thought
markers of suicidal subjects: A prospective multicenter trial. Suicide & Life-
Threatening Behavior, 47(1), 112–121.
Pospos, S., Young, I. T., Downs, N., Iglewicz, A., Depp, C., Chen, J. Y., … Zisook, S.
(2018). Web-based tools and mobile applications to mitigate burnout, depression,
and suicidality among healthcare students and professionals: A systematic review.
Academic Psychiatry, 42(1), 109–120.
Ren, F., Kang, X., & Quan, C. (2015). Examining accumulated emotional traits in suicide
blogs with an emotion topic model. IEEE Journal of Biomedical and Health Informatics,
20(5), 1384–1396.
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., &
Koffel, J. B. (2021). PRISMA-s: An extension to the PRISMA statement for reporting
literature searches in systematic reviews. Systematic Reviews, 10(1), 1–19.
Ribeiro, J. D., Franklin, J. C., Fox, K. R., Bentley, K. H., Kleiman, E. M., Chang, B. P., &
Nock, M. K. (2016). Self-injurious thoughts and behaviors as risk factors for future
suicide ideation, attempts, and death: A meta-analysis of longitudinal studies.
Psychological Medicine, 46(2), 225–236.
Roberts, A. E., Davenport, T. A., Wong, T., Moon, H.-W., Hickie, I. B., & LaMonica, H. M.
(2021). Evaluating the quality and safety of health-related apps and e-tools:
Adapting the mobile app rating scale and developing a quality assurance protocol.
Internet Interventions, 24, Article 100379.
RStudio Team. (2020). RStudio: Integrated development environment for r. RStudio, PBC.
http://www.rstudio.com/.
Runeson, B., Odeberg, J., Pettersson, A., Edbom, T., Adamsson, I. J., & Waern, M. (2017).
Instruments for the assessment of suicide risk: A systematic review evaluating the
certainty of the evidence. PLoS One, 12(7).
Sawhney, R., Joshi, H., Gandhi, S., Jin, D., & Shah, R. R. (2021). Robust suicide risk
assessment on social media via deep adversarial learning. Journal of the American
Medical Informatics Association, 28(7), 1497–1506.
Scherer, K. (1986). Vocal affect expression: A review and a model for future research.
Psychological Bulletin, 99(2), 143.
Scherer, S., Pestian, J., & Morency, L.-P. (2013). Investigating the speech characteristics of
suicidal adolescents (pp. 709–713).
Sierra, G., Andrade-Palos, P., Bel-Enguix, G., Osornio-Arteaga, A., Cabrera-Mora, A.,
García-Nieto, L., & Sierra-Aparicio, T. (2021). Suicide risk factors: A language
analysis approach in social media. Journal of Language and Social Psychology, 1–19.
Silverman, S. E., Silverman, M. K., et al. (2006). Methods and apparatus for evaluating
near-term suicidal risk using vocal parameters. U.S. Patent 7,062,443.
Song, J., Song, T. M., Seo, D.-C., & Jin, J. H. (2016). Data mining of web-based
documents on social networking sites that included suicide-related words among
korean adolescents. The Journal of Adolescent Health, 59(6), 668–673.
Stange, J. P., Kleiman, E. M., Mermelstein, R. J., & Trull, T. J. (2019). Using ambulatory
assessment to measure dynamic risk processes in affective disorders. Journal of
Affective Disorders, 259, 325–336.
Stasak, B., Epps, J., Schatten, H. T., Miller, I. W., Provost, E. M., & Armey, M. F. (2021).
Read speech voice quality and disfluency in individuals with recent suicidal ideation
or suicide attempt. Speech Communication, 132, 10–20.
Sung, J. E., Choi, S., Eom, B., Yoo, J. K., & Jeong, J. H. (2020). Syntactic complexity as a
linguistic marker to differentiate mild cognitive impairment from normal aging.
Journal of Speech, Language, and Hearing Research, 63(5), 1416–1429.
Szmrecsanyi, B. (2004). On operationalizing syntactic complexity. 2 pp. 1032–1039).
Tadesse, M. M., Lin, H., Xu, B., & Yang, L. (2020). Detection of suicide ideation in social
media forums using deep learning. Algorithms, 13(1), 7.
Teixeira, A. S., Talaga, S., Swanson, T. J., & Stella, M. (2021). Revealing semantic and
emotional structure of suicide notes with cognitive network science. Scientific
Reports, 11(1), 1–15.
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt-Jepsen, M., Whelan, P., …
Firth, J. (2021). The growing field of digital psychiatry: Current evidence and the
future of apps, social media, chatbots, and virtual reality. World Psychiatry, 20(3),
318–335.
Trotzke, A., & Bayer, J. (2015). Syntactic complexity across interfaces (Vol. 30). Walter de
Gruyter GmbH & Co KG.
Vallduvi, E. (1992). A preverbal landing site for quantificational operators. In Catalan
working papers in Linguistics (pp. 319–343).
Van Heeringen, K., Hawton, K., & Williams, J. M. G. (2000). Pathways to suicide: An
integrative approach. The International Handbook of Suicide and Attempted Suicide,
223–234.
Van Orden, K. A., Witte, T. K., Cukrowicz, K. C., Braithwaite, S. R., Selby, E. A., &
Joiner, T. E., Jr. (2010). The interpersonal theory of suicide. Psychological Review,
117(2), 575.
S. Homan et al.
Vioules, M. J., Moulahi, B., Azé, J., & Bringay, S. (2018). Detection of suicide-related
posts in twitter data streams. IBM Journal of Research and Development, 62(1), 1–7.
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D.
(2019). Standalone smartphone apps for mental health—A systematic review and
meta-analysis. Npj Digital Medicine, 2(1), 1–10.
Weldon, M., Poyade, M., Martin, J. L., Sharp, L., & Martin, D. (2019). Using interactive
3D visualisations in neuropsychiatric education. In Biomedical visualisation (pp.
17–27). Springer.
Witt, K., Spittal, M. J., Carter, G., Pirkis, J., Hetrick, S., Currier, D., … Milner, A. (2017).
Effectiveness of online and mobile telephone applications (‘apps’) for the self-
management of suicidal ideation and self-harm: A systematic review and meta-
analysis. BMC Psychiatry, 17(1), 297.
World Health Organization. (2020). Suicide worldwide in 2019 – Global health
estimates. https://www.who.int/publications/i/item/9789240026643. Last
accessed on 2020-12-12.
Xu, Z., Xu, Y., Cheung, F., Cheng, M., Lung, D., Law, Y. W., … Yip, P. S. (2021). Detecting
suicide risk using knowledge-aware natural language processing and counseling
service data. Social Science & Medicine, 283, 1–8.
19
Clinical Psychology Review 95 (2022) 102161
Yingthawornsuk, T., Keskinpala, H. K., France, D., Wilkes, D. M., Shiavi, R. G., &
Salomon, R. M. (2006). Objective estimation of suicidal risk using vocal output
characteristics. In Ninth International Conference on Spoken Language Processing.
Yingthawornsuk, T., Keskinpala, H. K., Wilkes, D. M., Shiavi, R. G., & Salomon, R. M.
(2007). Direct acoustic feature using iterative EM algorithm and spectral energy for
classifying suicidal speech. In Eighth Annual Conference of the International Speech
Communication Association.
Yingthawornsuk, T., & Shiavi, R. (2008). Distinguishing depression and suicidal risk in
men using GMM based frequency contents of affective vocal tract response.
Psychology. https://doi.org/10.1109/ICCAS.2008.4694621
Zhang, L., Duvvuri, R., Chandra, K. K., Nguyen, T., & Ghomi, R. H. (2020). Automated
voice biomarkers for depression symptoms using an online cross-sectional data
collection initiative. Depression and Anxiety, 37(7), 657–669.
Zhang, T., Schoene, A. M., & Ananiadou, S. (2021). Automatic identification of suicide
notes with a transformer-based deep learning model. Internet Interventions, 25,
Article 100422.
Zhong, Q.-Y., Karlson, E. W., Gelaye, B., Finan, S., Avillach, P., Smoller, J. W., …
Williams, M. A. (2018). Screening pregnant women for suicidal behavior in
electronic medical records: Diagnostic codes vs. clinical notes processed by natural
language processing. BMC Medical Informatics and Decision Making, 18(1), 1–11.