RESEARCH ARTICLE

Predicting Homelessness Among Transitioning U.S.

Army Soldiers
Jack Tsai, PhD,1,2 Dorota Szymkowiak, PhD,1 Dina Hooshyar, MD,1,3 Sarah M. Gildea, BS,4
Irving Hwang, MS,4 Chris J. Kennedy, PhD,5 Andrew J. King, MS,4 Katherine A. Koh, MD,5,6
Alex Luedtke, PhD,7,8 Brian P. Marx, PhD,9,10 Ann E. Montgomery, PhD,1,11,12
Robert W. O’Brien, PhD,13 Maria V. Petukhova, PhD,4 Nancy A. Sampson, BA,4
Murray B. Stein, MD,14,15,16 Robert J. Ursano, MD,17 Ronald C. Kessler, PhD4

Introduction: This study develops a practical method to triage Army transitioning service
members (TSMs) at highest risk of homelessness to target a preventive intervention.

Methods: The sample included 4,790 soldiers from the Study to Assess Risk and Resilience in
Servicemembers-Longitudinal Study (STARRS-LS) who participated in 1 of 3 Army STARRS
2011−2014 baseline surveys followed by the third wave of the STARRS-LS online panel surveys
(2020−2022). Two machine learning models were trained: a Stage-1 model that used administrative
predictors and geospatial data available for all TSMs at discharge to identify high-risk TSMs for initial
outreach; and a Stage-2 model estimated in the high-risk subsample that used self-reported survey data
to help determine highest risk based on additional information collected from high-risk TSMs once
they are contacted. The outcome in both models was homelessness within 12 months after leaving active
service.
Results: Twelve-month prevalence of post-transition homelessness was 5.0% (SE=0.5). The
Stage-1 model identified 30% of high-risk TSMs who accounted for 52% of homelessness. The
Stage-2 model identified 10% of all TSMs (i.e., 33% of high-risk TSMs) who accounted for 35% of
all homelessness (i.e., 63% of the homeless among high-risk TSMs).
Conclusions: Machine learning can help target outreach and assessment of TSMs for homeless
prevention interventions.
Am J Prev Med 2024;66(6):999−1007. Published by Elsevier Inc. on behalf of American Journal of Preventive
Medicine.

From the 1National Center on Homelessness among Veterans, VA Home-
less Programs Office, Washington, District of Columbia; 2School of Public
Health, University of Texas Health Science Center at Houston, Houston,
Texas; 3Department of Psychiatry, University of Texas Southwestern Medical
Center, Dallas, Texas; 4Department of Health Care Policy, Harvard Medical
School, Boston, Massachusetts; 5Department of Psychiatry, Massachusetts
General Hospital, Boston, Massachusetts; 6Boston Health Care for the
Homeless Program, Boston, Massachusetts; 7Department of Statistics, Uni-
versity of Washington, Seattle, Massachusetts; 8Vaccine and Infectious Dis-
ease Division, Fred Hutchinson Cancer Research Center, Seattle,
Washington; 9National Center for PTSD, VA Boston Healthcare System,
Boston, Massachusetts; 10Department of Psychiatry, Boston University Cho-
banian and Avedisian School of Medicine, Boston, Massachusetts; 11School
of Public Health, University of Alabama at Birmingham, Birmingham,
Alabama; 12VA Birmingham Health Care System, Birmingham, Alabama;
13
VA Health Services Research and Development Service, Washington, Dis-
trict of Columbia; 14Department of Psychiatry, University of California San
Diego, La Jolla, California; 15School of Public Health, University of California
San Diego, La Jolla, California; 16VA San Diego Healthcare System, San
Diego, California; and 17Department of Psychiatry, Center for the Study of
Traumatic Stress, Uniformed Services University of the Health Sciences,
Bethesda, Maryland
Address correspondence to: Jack Tsai, PhD, University of Texas
Health Science Center at Houston, 1200 Pressler Street, Houston, TX,
78240. E-mail: Jack.Tsai@uth.tmc.edu.
0749-3797/$36.00
https://doi.org/10.1016/j.amepre.2024.01.018

Published by Elsevier Inc. on behalf of American Journal of Preventive Medicine. Am J Prev Med 2024;66(6):999−1007 999

Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
1000 Tsai et al / Am J Prev Med 2024;66(6):999−1007
INTRODUCTION
H
omelessness among veterans is a major public
health issue that has taken considerable federal
and local healthcare, social services, and com-
munity resources to address.1,2 Prior research, although
furthering understanding of risk factors,3,4 was limited
in two ways. First, it was for the most part based on
retrospective rather than prospective data. Second, it
focused largely on individual-level risk factors (e.g.,
socio-demographics, clinical diagnoses) rather than
including additional information about community-level
risk factors.5
Both the Department of Defense (DoD) and the
Department of Veterans Affairs (VA) help service mem-
bers transition out of the military back into civilian life
when their term of service ends. The VA operates a
National Call Center for Homeless Veterans that is avail-
able 24 hours a day, 7 days a week to help veterans pre-
vent homelessness and is an important resource for
connecting veterans with needed healthcare and social
services.6 However, most callers are already homeless or
at imminent risk of homelessness and so the Call Center
is mostly focused on secondary prevention. More
upstream interventions focused on primary prevention
are needed. Recent legislation—the Solid Start Act of
20227—requires DoD and VA to coordinate efforts to
assist transitioning service members (TSMs) in accessing
benefits. This is critical to do early in the transition pro-
cess given data, quite different from earlier service eras,8
that TSMs who served during the Iraq/Afghanistan con-
flicts often experience homelessness within one year of
military discharge.9,10 This highlights the great potential
value of identifying TSMs at highest risk for homeless-
ness for early prevention efforts.
The Study to Assess Risk and Resilience in Service-
members Longitudinal Study (STARRS-LS) extends the
efforts of the original Army STARRS project to follow
soldiers after leaving active service.11 A model developed
with STARR-LS data accessed before leaving active
service had good accuracy predicting subsequent self-
reported homelessness.10 However, this model only
included a few geographic and community variables from
the STARR-LS database and did not specifically examine
homelessness within 12 months after leaving service.
Nonetheless, the success of the initial STARRS-LS
model led to a collaboration with the VA Homeless Pro-
grams to implement a homelessness prevention inter-
vention with TSMs. This intervention, known as
HEARTH (Help with Employment, Agency, Risk, Tran-
sitions, and Housing), is currently in operation and uses
the targeting models described in the current report to
determine which TSMs to reach out to.
Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
METHODS
Study Sample
STARRS-LS is a longitudinal epidemiological-neurobio-
logical study designed to evaluate risk and protective
factors for behavioral health problems among Army
soldiers.11 Field procedures have been detailed
elsewhere.12,13 There were three baseline Army STARRS
surveys: (1) New Soldier Study (2011−2012, n=38,733
soldiers); (2) All Army Study (AAS, 2011−2013,
n=25,088); and (3) Pre-Post Deployment Study (2012
−2014, n=8,566 soldiers). All participants provided writ-
ten informed consent. The Human Subjects Committees
of the University of Michigan, the Uniformed Services
University of the Health Sciences, and the Army Medical
Research and Materiel Command approved all recruit-
ment, consent, and field procedures. The n=72,387 sur-
vey respondents who consented to link their deidentified
survey data with Army administrative data were the
focus of this study. As detailed elsewhere,14 calibration
weights were applied to these cases to adjust for differen-
ces in survey responses between respondents who did
versus did not agree to administrative linkage between
the population and the sample on a wide range of
administrative variables.
As described in Appendix A and elsewhere,15 three
subsequent STARRS Longitudinal Surveys (LS) were
carried out in 2016−2018 (LS1), 2018−2019 (LS2), and
2020−2022 (LS3) in a probability subsample of baseline
STARRS survey respondents. LS1 (n=14,508, 35.6%
response rate in the baseline sample of STARRS
respondents weighted to adjust for baseline nonresponse
with respect to known Army administrative variables)
oversampled soldiers in the baseline surveys with a his-
tory of either mental disorder or suicidality, those with
special operations duty assignments, those who self-
identified as female, and members of the activated Army
National Guard and Army Reserve. The LS1 data were
reweighted to adjust for this differential sampling. LS1
respondents were then resurveyed in LS2 (n=12,156,
83.7% conditional response rate among LS1 respond-
ents) and LS3 (n=11,119, 76.7% conditional response
rate among LS1 respondents).
The analyses described in the current report focused
on the 4,790 LS3 respondents in the Regular Army at
baseline who reported at LS3 no longer being in active
service or in a Reserve or National Guard Component,
on orders, or activated. A 70% randomly selected train-
ing sample (n=3,352) was selected to build the Stage-1
model using Army administrative data available for all
soldiers at the time of leaving active service.
The rarity of homelessness raised a concern about
possible model bias toward the majority class. Numerous
www.ajpmonline.org
 Tsai et al / Am J Prev Med 2024;66(6):999−1007
relatively equivalent methods have been proposed to
address this problem.16 One of the most common of
these methods was used: a case-control subsampling
scheme that included 100% of the n=227 respondents in
the training sample who became homeless plus a proba-
bility sample of controls 5 times as large (n=1,135). Each
respondent who became homeless was assigned a weight
of 5 and each control a weight of 1 so that exactly 50%
of weighted observations experienced homelessness. The
number of controls per case was set at 5 based on the
fact that there is little gain in statistical power when the
control to case ratio is greater than 5:1.17
Based on Stage-1 model results, a subsample of TSMs
in the original sample was defined as high-risk for home-
lessness. A Stage-2 model was then estimated in that
subsample using a composite predicted probability of
homelessness from the Stage-1 model in addition to
STARRS survey variables. The rationale for this was that
a brief survey could be administered to the high-risk
TSMs contacted by HEARTH social workers to offer
transitional assistance as a way of distinguishing between
highest-risk and other high-risk TSMs. The Stage-2
model was designed to select the variables for that brief
survey.
Measures
LS3 respondents were asked if they had ever been home-
less since leaving Regular Army service. Homelessness
was defined as “not having stable housing that you either
own, rent, or stay in as part of a household.” Follow-up
questions gathered information about episodes and tim-
ing of homelessness.
A total of 38 potential predictors were selected from
the VA/DoD Identity Repository (VADIR) for use in the
Stage-1 model. VADIR is a database provided by the
DoD to VA for all TSMs within 1 week of leaving active
service. VADIR includes basic information about socio-
demographics and career characteristics (e.g., age of
enlistment, rank, military occupational specialty, deploy-
ments, disability rating) along with TSMs’ postmilitary
contact information (e.g., phone numbers, addresses).
In addition, the zip codes of TSMs were linked to
community-level data at the zip code-level and aggre-
gated county-level data using a block group to zip code
crosswalk. This resulted in 171 geospatial variables char-
acterizing the area where each TSM moved after leaving
active service.
Building on the Stage-1 model, STARRS survey varia-
bles were used to develop the Stage-2 model. A total of
92 variables hypothesized to be potential predictors of
homelessness were extracted from the LS survey data-
base for this purpose. These included socio-demo-
graphics, Army career characteristics, personality
June 2024
Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
1001
characteristics, information about adverse childhood
experiences and other stressors, self-injurious thoughts
and behaviors, physical disorders, and mental disorders.
A detailed listing is available from the corresponding
author.
Statistical Analysis
The Random Forest (RF) algorithm18 was used to train
the Stage-1 model. RF is an ensemble regression tree
method that estimates many trees (2,000 in the current
study) based on random subsets of predictors applied to
random subsets of cases and pooled estimates across the
ensemble to arrive at a stable prediction of an outcome.
RF was implemented in the ranger R program.19 Key
hyperparameter values (minimum node size, maximum
depth, number of predictors) were varied to optimize
area under the receiver operating curve (AU-ROC). A
simple lasso penalized regression model20 was also esti-
mated as a benchmark to evaluate how much the addi-
tion of nonlinear and nonadditive information in RF
improves predictions over a simple linear-additive
model. No more complex algorithms than RF were used
in Stage-1, given the minimum set of VADIR predictors
available for estimating the Stage-1 model.
All 92 potential survey predictors along with a com-
posite predicted probability of homelessness developed
from the Stage-1 model were used to train the Stage-2
model in the entire high-risk sample (i.e., combining the
training and test samples). Analysis began by estimating
a univariable 5-fold cross-validated (CV) AU-ROC for
each of the 92 potential Stage-2 predictors. Further anal-
ysis then focused on the subset of these variables that
had univariable CV-AU-ROCs with a lower bound 95%
CI of 0.51 or greater. Many of these variables were
multi-item scales. Rather than use these scales as predic-
tors in the Stage-2 model, the items in the scales were
entered as separate predictors, with the goal of develop-
ing a brief survey that used only the most important of
these predictors.
Although the Stage-2 model, like the Stage-1, used RF,
more complex algorithms were also considered for Stage-
2 with the Super Learner (SL) ensemble machine learning
method. SL pools estimates across multiple algorithms
(“ensemble”) using a weight guaranteed in expectation to
generate a pooled prediction at least as accurate as the
best component algorithm in the ensemble.21 Consistent
with recommendations,22 a diverse set of algorithms was
used in the SL ensemble to capture nonlinearities and
interactions and reduce risk of misspecification (Appen-
dix Table 1, available online).23 A parallel lasso model
was estimated as a benchmark.
As it would be difficult to compute a Stage-2 model
prediction in real time for a single respondent if the
1002 Tsai et al / Am J Prev Med 2024;66(6):999−1007
model was anything other than linear-additive, an
attempt was made to fit a lasso model with the predictors
that emerged as important from the more complex mod-
els along with all possible 2-way interactions among
those variables.
Predictor importance in estimating the ranger was
examined using the kernel Shapley Additive Explana-
tions (SHAP) method24 implemented in the fastshap R
package.25 This method estimates the effect of changing
a predictor from its observed score to the sample mean
averaged across all logically possible permutations of
other predictors. Proportional mean absolute SHAP val-
ues (SHAPP) were calculated by dividing mean absolute
SHAP values of important predictors by the mean abso-
lute SHAP value of the entire model. A bee swarm plot
was used to identify dominant directions and distribu-
tions of associations.
Data management and calculation of prevalence and
AU-ROCs were implemented in SAS version 9.4. The
ranger, lasso, and SHAP value analyses were imple-
mented in R version 3.6.3.
RESULTS
In the total study sample (n= 4,790), the prevalence of
homelessness within 12 months of leaving active service
was 5.0% (standard error=0.5). Appendix Table 2 (avail-
able online) shows the demographic and background
characteristics of the study sample.
For the Stage-1 model (VADIR plus geospatial predic-
tors), the best model in the training sample had an AU-
ROC of 0.65 in the test sample. The optimal hyperpara-
meters for that model were minimum node size=30,
maximum depth=3, and number of predictors=unres-
tricted. The dominant direction, proportional SHAP,
and bee swarm plot (to concretize information about
dominant direction) for the 10 predictors that emerged
as important in that model are displayed in Figure 1.
The bee swarms show that most predictors have domi-
nant directions, but that two (time in service and County
child food insecurity rate) have mixed signs depending
on individual TSM characteristics. Proportional SHAPs
sum to more than 100% because of strong inter-correla-
tions among some predictors. The county unemploy-
ment rate where the TSM resided, age of the TSM,
duration of service in the Army, and percentage of adults
75 years or older in the county were the most important
predictors. Geospatial characteristics constituted 6 of the
10 predictors.
The model was evaluated in the test sample (Table 1),
which found high proportional homelessness concen-
trated in the top 30% of the sample (first, third, fifth,
and sixth ventiles) accounting for 51.9% of all homeless-
ness. Based on this result, the 30% of TSMs with highest
predicted risk in the Stage-1 model were designated
“high risk.”
The Stage-2 model was then estimated within this
30% high-risk sample. Analysis began by calculating uni-
variable 5-fold CV-AU-ROC for each of the 92 potential
predictors with homelessness. Twenty-five of these pre-
dictors had CV-AU-ROCs with lower bound 95% CI
greater than or equal to 0.51 (Appendix Table 3, avail-
able online). Ten of these 25 were multi-item scales.
After disaggregating those scales into items, a total of 54
survey variables plus the composite predicted probability
of homelessness from the Stage-1 model were entered as
potential predictors in the Stage-2 model.
The best Stage-2 model was a ranger with the same
hyperparameters as the Stage-1 model (i.e., minimum
node size=30, maximum depth=3, and number of pre-
dictors=unrestricted). CV-AU-ROC was 0.72. The Super
Learner had a lower CV-AU-ROC (0.71) than the
ranger, indicating that all stable nonlinearities and non-
additivities in the data were captured by ranger. A total

Figure 1. SHAP values and bee swarm plots from the Stage-1 Random Forest model predicting homelessness Note: The mean
absolute value SHAP for all predictors was 0.058531. SHAP values were derived from a Random Forest prediction model. Abbrevia-
tions: SHAP, Shapley Additive Explanations
a
The predictors include 6 geospatial variables and 4 VADIR (Veterans Affairs/Department of Defense Identity Repository) variables.

www.ajpmonline.org

Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
 Tsai et al / Am J Prev Med 2024;66(6):999−1007 1003
a
Table 1. Stage-1 Random Forest Model of Homelessness in the 30% Test Sample (n=1,438, cases=97)

Sensitivity (SN)b Positive predictive value (PPV)c

Within-ventile Cumulative Within-ventile Cumulative
Risk ventile SN (SE) SN (SE) PPV (SE) PPV (SE)
1 - Top 5% 10.8 (4.0) 10.8 (4.0) 14.9 (5.3) 10.8 (4.0)
2 4.0 (2.2) 14.8 (4.4) 5.6 (3.4) 10.2 (3.4)
3 10.3 (5.7) 25.2 (6.3) 13.9 (8.1) 11.5 (3.5)
4 4.2 (3.0) 29.3 (6.7) 5.9 (4.2) 10.1 (2.9)
5 12.1 (5.2) 41.5 (6.8) 16.8 (6.9) 11.5 (2.7)
6 10.4 (4.3) 51.9 (6.7) 14.7 (5.7) 12.0 (2.5)
7 3.9 (1.6) 55.8 (6.7) 5.2 (2.2) 11.0 (2.2)
8 7.0 (2.4) 62.7 (6.4) 10.0 (3.6) 10.9 (2.0)
9 2.1 (1.3) 64.8 (6.4) 2.9 (1.5) 10.0 (1.7)
10 6.4 (4.7) 71.2 (5.2) 8.3 (6.2) 9.8 (1.7)
11 6.7 (1.3) 77.9 (4.8) 10.0 (2.0) 9.8 (1.6)
12 2.0 (1.0) 79.9 (4.8) 2.7 (1.4) 9.2 (1.5)
13 3.4 (2.8) 83.3 (3.9) 4.6 (3.8) 8.9 (1.3)
14 2.4 (0.3) 85.7 (3.8) 3.4 (0.5) 8.5 (1.2)
15 6.5 (2.9) 92.2 (2.7) 8.9 (4.0) 8.5 (1.2)
16 1.0 (0.9) 93.2 (2.5) 1.4 (1.3) 8.1 (1.1)
17 2.4 (1.7) 95.5 (1.9) 3.3 (2.4) 7.8 (1.1)
18 2.4 (1.4) 98.0 (1.5) 3.4 (2.1) 7.5 (1.0)
19 1.3 (1.3) 99.3 (0.7) 1.8 (1.8) 7.2 (1.0)
20 - Bottom 5% 0.7 (0.7) 100.0 (0.0) 1.0 (1.0) 6.9 (0.9)
a
See Figure 1 for a listing of the 10 predictors in the Stage-1 model, which was estimated in the 70% training case-control sample and tested in the
30% test sample. The n=1,438 respondents in the test sample included n=97 of the n= 324 total-sample respondents who reported homelessness
12 months after separating or retiring from the Regular Army. Area under the receiver operating curve (AU-ROC) of the model in the test sample was
0.65.
b
Percent of all observed homelessness within or cumulative up through a given ventile of the sample.
c
The proportion of respondents within or cumulative up through a given ventile who were homeless in the first 12 months after separating or retiring
from the Regular Army.
SE, standard error of SN or PPV.

of 10 survey predictors plus the composite predicted
probability were important features in the Stage-2 ranger
model (Appendix Table 4, available online). It was an
easy matter to create a brief survey to administer these
questions in initial contacts with Stage-1 high-risk
TSMs.
It was more challenging to determine how to implement
real-time scoring of the brief survey results to distinguish
between highest-risk and other high-risk TSMs given that
the ranger was based on 2,000 decision trees, each with
numerous terminal nodes. As noted in the section on anal-
ysis methods, this problem was addressed by estimating a
lasso model using the 10 identified survey predictors, the
composite variable for predicted probability of homeless-
ness from the Stage-1 model (which could be created prior
to contacting the high-risk TSM and preloaded into the
brief survey software), and all possible 2-way interactions
among these predictors. This lasso model had a CV-AU-
ROC of 0.71, which was comparable to the CV-AU-ROC
of the ranger model, suggesting that the higher-order
interactions captured by the ranger model led to over-fit-
ting. The 17 predictors that emerged in the lasso model
included the composite predicted probability of homeless-
ness from the Stage-1 model, 8 of the 10 survey variables,
and 8 two-way interactions (Table 2). Predictors with the
largest coefficients included the composite predicted proba-
bility, any lifetime suicide attempt prior to leaving active
service, the interaction between lifetime suicidal ideation
before leaving service and the composite predicted proba-
bility, and the interaction between having any young
dependent children with being moderately/very religious.
An evaluation of lasso model performance showed
that the top 2 ventiles of the total sample (i.e., the top
33.3% of the 30% of TSMs defined by the Stage-1 model
as high-risk) had elevated risk and accounted for 67.2%
of all homelessness in the high-risk subsample (Table 3).
This means that the 10% of the total sample (i.e., 33.3%
of 30%) identified in the Stage-2 model accounted for
34.9% (i.e., 67.2% of 51.9%) of all the homelessness in
the total sample.

June 2024

Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
1004 Tsai et al / Am J Prev Med 2024;66(6):999−1007
Table 2. Cross-Validated Stage-2 Lasso Model in the 30% DISCUSSION
High-Risk Sample (n=1,567, cases=251)a
The analysis found that 5% of TSMs experienced home-
Predictors Lasso coefficients lessness within one year of leaving active service (com-
Composite quartile 2 or aboveb 1.01519 pared to 0.18% in the entire U.S. population).5 Through
1+ lifetime suicide attemptc 0.81041 a series of machine learning models applied to a compre-
Lifetime suicidal ideationd x composite 0.21410 hensive dataset of individual- and community-level
quartile 3 or aboveb characteristics of TSMs, parsimonious models were
Any dependents ages 0-5 x moderately 0.79338 developed that could identify the highest-risk group of
or very religiouse
Lifetime passive suicidal ideationc 0.17385 these TSMs. Although the AU-ROCs of these models
Any dependents ages 0-17 x 0.22485 were in the range typically thought of as modest,26 the
moderately or very religiouse top 30% of high-risk TSMs identified in the Stage-1
Any dependents ages 0-17 x felt sad or 0.07199 model nonetheless accounted for the majority (52%) of
depressed some of the time or moref all homelessness experienced in the sample and the 33%
Lifetime active suicidal ideationc x 0.03428
of the high-risk TSMs identified in the Stage-2 model as
composite quartile 3 or aboveb
Lifetime suicidal ideationd 0.22460 highest risk accounted for two-thirds of all homeless in
Any dependents ages 0-5 x composite 0.02940 the high-risk subsample.
quartile 4b Based on these results, the Stage-1 model is currently
Any dependents ages 0-5 x very 0.24666 being used to determine which TSMs to contact in the
religiouse
VA HEARTH intervention, a wrap-around intervention
1+ lifetime suicide attemptc x Felt sad 0.36282
or depressed a lot of the time or moref
based on the validated Critical Time Intervention model
Felt sad or depressed some of the time 0.05275 for homelessness prevention.27,28 VA social workers are
or moref x Very religiouse contacting these high-risk TSMs within one month of
2+ lifetime suicide attemptsc x 0.42686 leaving service to offer help with a wide range of transi-
moderately or very religiouse tional services focused on preventing homelessness. The
1+ lifetime suicide attemptc x very 1.04589
religiouse brief survey used to generate predictions based on the
Felt sad or depressed a lot of the time 0.15159 Stage-2 model is administered as part of the first contact.
or moref x Very religiouse Results are scored in real time and used by the social
Any dependents ages 0-17 x lifetime 0.05071 workers to determine the intensity of recontacts to pro-
active suicidal ideationc pose to the TSM before closing the first contact. The
a
The composite predicted probability of homelessness converted to
nested dichotomies plus 8 survey variables (5 dichotomies and 3 cate-
STARRS-LS prediction models, incorporating as they do
gorical variables converted to nested dichotomies). All variables were community-level factors, are an important improvement
defined as of separation or retirement from the Regular Army (e.g., on previous models.3,4,29−34 At the community-level,
administrative or medical discharge or retirement, fulfilled service obli-
gation, reached retired pay eligibility age or 20+ years of qualifying ser- higher unemployment rates, proportion of aging adults,
vice, released from obligation, transferred to Individual Ready Reserve, and proportion of widowed adults in the region are asso-
Inactive National Guard, or Standby Reserve). ciated with increased risk for homelessness. At the indi-
b
Quartiles of the composite are defined within the top 30% of the
composite. vidual level, history of suicidality is a major predictor of
homelessness, which is consistent with findings from
c
At any survey up to and including the last STARRS survey the Soldier
took before they separated or retired from the Regular Army, either a
baseline STARRS survey (i.e., All Army Study, New Soldier Study, or Pre- past studies.35,36
Post Deployment Study survey) or a STARRS-Longitudinal Study (LS)
survey.
d
Limitations
Either active (had thoughts of killing self) or passive (wanted to be
dead or go to sleep and never wake up) suicidal ideation at any survey
This study had several noteworthy limitations. First, the
up to and including the last STARRS survey the Soldier took before they sample was restricted to soldiers who participated in the
separated or retired from the Regular Army, either a baseline STARRS third wave of STARRS-LS, so there is a possibility of
survey or a STARRS-LS survey.
e
Soldiers were asked to indicate how religious they consider them- sample bias despite the use of adjustment weights. Sec-
selves, with response options of very, moderately, slightly, and not at all. ond, the coding of geospatial variables relied on the
f
Soldiers were asked to focus on the 1 month in their life when they had accuracy of addresses in the Army administrative data
the most problems with feeling sad, depressed, or hopeless. They were
then asked how often during that month they had each of a series of system. These addresses are not always accurate. Third,
symptoms. One of those symptoms was “feeling sad or depressed.” analysis focused on developing a parsimonious assess-
Response categories were none of the time, a little of the time, some of
the time, most of the time, and all or almost all the time. This item was ment tool for the highest-risk soldiers, but broader mod-
taken from their last survey before separating/retiring from the Regular els could be developed to target a wider range of risk if
Army. adequate resources were provided. Fourth, the

www.ajpmonline.org

Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
 Tsai et al / Am J Prev Med 2024;66(6):999−1007 1005
a
Table 3. Cross-Validated Stage-2 Lasso Model in the 30% High-Risk Sample (n=1,567, cases=251)

Sensitivity (SN)b Positive predictive value (PPV)c

Within-sextile Cumulative Within-sextile Cumulative
Risk sextile % (SE) % (SE) % (SE) % (SE)
1 - Top 16.66% 19.7 (2.1) 19.7 (2.1) 44.1 (4.2) 44.1 (4.2)
2 15.2 (2.2) 34.9 (3.0) 30.3 (4.5) 36.8 (3.1)
3 7.6 (1.4) 42.5 (3.3) 19.7 (3.7) 31.8 (2.5)
4 5.6 (1.5) 48.1 (3.6) 15.4 (3.9) 28.3 (2.2)
5 3.7 (1.0) 51.9 (3.8) 4.9 (1.3) 21.0 (1.6)
6 - Bottom 16.66% 0.0 (0.0) 51.9 (3.8) 0.1 (0.1) 20.4 (1.6)
a
See Table 2 for a list of the predictors in the Stage-2 model, which was estimated in the 70% training case-control sample and tested in the 30% test
sample. The n=1,567 respondents in the total sample defined as high-risk by the Stage-1 model were the sample considered here. This sample
included n=251 respondents who reported homelessness 12 months after separating or retiring from the Regular Army. Given the small size of the
high-risk sample, cross-validation was used rather than separate training and test samples to estimate SN and PPV.
b
Percent of all observed homelessness within or cumulative up through a given ventile of the sample.
c
The proportion of respondents within or cumulative up through a given ventile who were homeless in the first 12 months after separating or retiring
from the Regular Army.
VADIR, Veterans Affairs/Department of Defense Identity Repository; SE, standard error of SN or PPV.

administrative variables in the Stage-1 model were
restricted to the limited set in the VADIR dataset. The
Stage-1 could be improved in future if the Army devel-
oped the capacity for real-time integration and access
to their much more extensive set of administrative
databases.
CONCLUSIONS
Development of a predictive analytics model that identi-
fies TSMs at high risk of homelessness within 12 months
of leaving the military advances two overarching goals of
public health: protecting the health of entire populations
and preventing problems from happening or reoccur-
ring. The analysis reported here demonstrated that such
a model can be built based on existing Army administra-
tive data and a 10-question self-assessment. This work
has already been useful in guiding outreach efforts in a
pilot homelessness prevention intervention. Depending
on the results of the forthcoming evaluation of that
intervention, the model might stimulate the develop-
ment of more refined models and more expansive proac-
tive interventions.
ACKNOWLEDGMENTS
The Army STARRS Team consists of Co-Principal Investigators:
Robert J. Ursano, MD (Uniformed Services University) and Mur-
ray B. Stein, MD, MPH (University of California San Diego and
VA San Diego Healthcare System); Site Principal Investigators:
James Wagner, PhD (University of Michigan) and Ronald C.
Kessler, PhD (Harvard Medical School); Army scientific consul-
tant/liaison: Kenneth Cox, MD, MPH (Office of the Assistant
Secretary of the Army (Manpower and Reserve Affairs); and
Other team members: Pablo A. Aliaga, MA (Uniformed Services
University); David M. Benedek, MD (Uniformed Services
University); Laura Campbell-Sills, PhD (University of California
San Diego); Carol S. Fullerton, PhD (Uniformed Services Univer-
sity); Nancy Gebler, MA (University of Michigan); Meredith
House, BA (University of Michigan); Paul E. Hurwitz, MPH (Uni-
formed Services University); Sonia Jain, PhD (University of Cali-
fornia San Diego); Tzu-Cheg Kao, PhD (Uniformed Services
University); Lisa Lewandowski-Romps, PhD (University of Michi-
gan); Alex Luedtke, PhD (University of Washington and Fred
Hutchinson Cancer Research Center); Holly Herberman Mash,
PhD (Uniformed Services University); James A. Naifeh, PhD (Uni-
formed Services University); Matthew K. Nock, PhD (Harvard
University); Victor Puac-Polanco, MD, DrPH (Harvard Medical
School); Nancy A. Sampson, BA (Harvard Medical School); and
Alan M. Zaslavsky, PhD (Harvard Medical School).
As a cooperative agreement, scientists employed by the
National Institute of Mental Health and U.S. Army liaisons and
consultants collaborated to develop the study protocol and
data collection instruments, supervise data collection, interpret
results, and prepare reports. Although a draft of the manuscript
was submitted to the U.S. Army and National Institute of Mental
Health for review and comment before submission for publica-
tion, this was done with the understanding that comments
would be no more than advisory. The contents are solely the
responsibility of the authors and do not necessarily represent
the views of the NIMH, the Department of the Army, the Depart-
ment of Defense, or the Department of Veteran Affairs.
The project was supported by the U.S. Department of Veter-
ans Affairs (Contract number 36C24123C0030). Dr. Marx
was supported by VA Suicide Prevention Center grant
I01CX002621-01. Army STARRS was sponsored by the Depart-
ment of the Army and funded under cooperative agreement
number U01MH087981 (2009-2015) with the National Insti-
tute of Mental Health (NIMH). Subsequently, STARRS-LS was
sponsored and funded by the Department of Defense (USUHS
grant number HU0001-15-2-0004).
In the past 3 years, Dr. Kessler was a consultant for Cam-
bridge Health Alliance, Canandaigua VA Medical Center, Child
Mind Institute, Holmusk, Massachusetts General Hospital,
Partners Healthcare, Inc., RallyPoint Networks, Inc., Sage

June 2024

Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
1006 Tsai et al / Am J Prev Med 2024;66(6):999−1007
Therapeutics and University of North Carolina. He has stock
options in Cerebral Inc., Mirah, PYM (Prepare Your Mind), Roga
Sciences and Verisense Health. In the past 3 years, Dr. Stein
received consulting income from Actelion, Acadia Pharmaceuti-
cals, Aptinyx, atai Life Sciences, Boehringer Ingelheim, Bionom-
ics, BioXcel Therapeutics, Clexio, EmpowerPharm, Engrail
Therapeutics, GW Pharmaceuticals, Janssen, Jazz Pharmaceuti-
cals, and Roche/Genentech. He has stock options in Oxeia Bio-
pharmaceuticals and EpiVario. He is paid for his editorial work
on Biological Psychiatry (Deputy Editor) and UpToDate (Co-Edi-
tor-in-Chief for Psychiatry). Dr. Goodman is a consultant for
Boehringer Ingelheim Pharmaceuticals.
No other financial disclosures have been reported by the
authors of this paper.
CREDIT AUTHOR STATEMENT
Jack Tsai: Conceptualization, Supervision, Writing − original
draft, Writing − review & editing. Dorota Szymkowiak: Writing −
review & editing. Dina Hooshyar: Conceptualization, Funding
acquisition, Project administration, Supervision, Writing −
review & editing. Sarah M. Gildea: Investigation, Project admin-
istration, Writing − review & editing. Irving Hwang: Formal anal-
ysis, Writing − review & editing. Chris J. Kennedy: Formal
analysis, Supervision, Writing − review & editing. Andrew J.
King: Formal analysis, Writing − review & editing. Katherine A.
Koh: Project administration, Writing − review & editing. Alex
Luedtke: Formal analysis, Supervision, Writing − review & edit-
ing. Brian P. Marx: Funding acquisition, Project administration,
Writing − review & editing. Ann E. Montgomery: Project admin-
istration, Writing − review & editing. Robert W. O’Brien: Project
administration, Writing − review & editing. Maria V. Petukhova:
Formal analysis, Writing − review & editing. Nancy A. Sampson:
Investigation, Project administration, Supervision, Writing −
review & editing. Murray B. Stein: Funding acquisition, Investi-
gation, Supervision, Writing − review & editing. Robert J.
Ursano: Funding acquisition, Investigation, Supervision, Writing
− review & editing. Ronald C. Kessler: Funding acquisition,
Investigation, Supervision, Writing − original draft, Writing −
review & editing.
SUPPLEMENTAL MATERIAL
Supplemental materials associated with this article can be
found in the online version at https://doi.org/10.1016/j.
amepre.2024.01.018.
REFERENCES
1. Tsai J, Pietrzak RH, Szymkowiak D. The problem of veteran homeless-
ness: an update for the new decade. Am J Prev Med. 2021;60(6):774–
780. https://doi.org/10.1016/j.amepre.2020.12.012.
2. Tsai J. Homelessness Among U.S. Veterans: Critical Perspectives. New
York, NY: Oxford University Press, 2018.
3. Tsai J, Rosenheck RA. Risk factors for homelessness among U.S. Vet-
erans. Epidemiol Rev. 2015;37(1):177–195. https://doi.org/10.1093/epi-
rev/mxu004.
4. Balshem H, Christensen V, Tuepker A. A critical review of the litera-
ture regarding homelessness among veterans. VA-ESP Project #05-
Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
225. https://www.hsrd.research.va.gov/publications/esp/homelessness.
pdf. Accessed June 26, 2023.
5. Sousa TD, Andrichik A, Cuellar M, Marson J, Prestera E, Rush K. The
2022 Annual Homelessness Assessment Report (AHAR) to Congress.
https://www.huduser.gov/portal/sites/default/files/pdf/2022-ahar-
part-1.pdf. Accessed September 5, 2023.
6. Tsai J, Montgomery AE, Szymkowiak D. Preventing homelessness
through the National Call Center for Homeless Veterans: Analysis of
calls and service referrals. Psychiatr Serv. 2023;74(3):316–319. https://
doi.org/10.1176/appi.ps.20220124.
7. Hassan MW. Public law no: 117-205 (10/17/2022): Solid Start Act of
2022. Summary: S.1198 — 117th Congress (2021-2022). https://www.
congress.gov/bill/117th-congress/senate-bill/1198#:~:text=This/20act/
20permanently/20authorizes/20and,of/20separation/20from/20the/
20military.&text=prioritize%20outreach%20to/20veterans%20who,
separation/20from/20the/20Armed/20Forces. Accessed September 8,
2023, 2023.
8. Tsai J, Szymkowiak D, Pietrzak RH. Delayed homelessness after mili-
tary discharge: examination of a sleeper effect. Am J Prev Med.
2020;59(1):109–117. https://doi.org/10.1016/j.amepre.2020.03.001.
9. Metraux S, Clegg LX, Daigh JD, Culhane DP, Kane V. Risk factors for
becoming homeless among a cohort of veterans who served in the era
of the Iraq and Afghanistan conflicts. Am J Public Health. 2013;103
(S2):S255–S261. https://doi.org/10.2105/AJPH.2013.301432.
10. Koh KA, Montgomery AE, O’Brien RW, et al. Predicting homeless-
ness among U.S. Army soldiers no longer on active duty. Am J Prev
Med. 2022;63(1):13–23. https://doi.org/10.1016/j.amepre.2021.12.028.
11. Ursano RJ, Colpe LJ, Heeringa SG, et al. The Army Study to Assess
Risk and Resilience in Servicemembers (Army STARRS). Psychiatry.
2014;77(2):107–119. https://doi.org/10.1521/psyc.2014.77.2.107.
12. Heeringa SG, Gebler N, Colpe LJ, et al. Field procedures in the Army
Study to Assess Risk and Resilience in Servicemembers (Army
STARRS). Int J Methods Psychiatr Res. 2013;22(4):276–287. https://
doi.org/10.1002/mpr.1400.
13. Kessler RC, Colpe LJ, Fullerton CS, et al. Design of the Army Study to
Assess Risk and Resilience in Servicemembers (Army STARRS). Int J
Methods Psychiatr Res. 2013;22(4):267–275. https://doi.org/10.1002/
mpr.1401.
14. Kessler RC, Heeringa SG, Colpe LJ, et al. Response bias, weighting
adjustments, and design effects in the Army Study to Assess Risk and
Resilience in Servicemembers (Army STARRS). Int J Methods Psy-
chiatr Res. 2013;22(4):288–302. https://doi.org/10.1002/mpr.1399.
15. Stanley IH, Chu C, Gildea SM, et al. Predicting suicide attempts
among U.S. Army soldiers after leaving active duty using information
available before leaving active duty: results from the Study to Assess
Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-
LS). Mol Psychiatry. 2022;27(3):1631–1639. https://doi.org/10.1038/
s41380-021-01423-4.
16. Rout N, Mishra D, Mallick MK. Handling imbalanced data: a survey.
In: Reddy MS, Viswanath K, SP KM, eds. In: International Proceed-
ings on Advances in Soft Computing, Intelligent Systems and Applica-
tions. Singapore. Springer Singapore, 2018:431–443.
17. Lachin JM. Biostatistical Methods: The Assessment of Relative Risks.
Hoboken, NJ: John Wiley & Sons, Inc., 2009.
18. Breiman L. Random forests. Mach Learn. 2001;45:5–32. https://doi.
org/10.1023/A:1010933404324.
19. Wright MN, Wager S, Probst P. R Package ’ranger’. https://cran.r-
project.org/web/packages/ranger/ranger.pdf. Accessed September 1,
2023, 2023.
20. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat
Soc Series B Stat Methodol. 1996;58(1):267–288.
21. Polley EC, Rose S, Van der Laan MJ. Super learning. In: van der
Laan MJ, Rose S, eds. Targeted Learning: Causal Inference for Obser-
vational and Experimental Data. 1st ed. New York, NY: Springer Pub-
lishing, 2011:44–66.
www.ajpmonline.org
 Tsai et al / Am J Prev Med 2024;66(6):999−1007
22. LeDell E, van der Laan MJ, Petersen M. AUC-maximizing ensembles
through metalearning. Int J Biostat. 2016;12(1):203–218. https://doi.
org/10.1515/ijb-2015-0035.
23. Kennedy CJ. Guide to SuperLearner. https://cran.r-project.org/web/
packages/SuperLearner/vignettes/Guide-to-SuperLearner.html. Accessed
November 27, 2023, 2023.
24. Lundberg SM, Lee SI. A unified approach to interpreting model pre-
dictions. In: Paper presented at: Proceedings of the 31st International
Conference on Neural Information Processing Systems; 2017. https://
papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-
Abstract.html. Accessed June 25, 2022.
25. GB. Fastshap: Fast approximate shapley values. https://cran.r-project.
org/web/packages/fastshap/index.html. Accessed June 25, 2022, 2022.
26. Hosmer DWJ, Lemeshow S, Sturdivant RX. Applied Logistic Regres-
sion: Third Edition. Hoboken, NJ: John Wiley & Sons, Inc., 2013.
27. Herman DB, Mandiberg JM. Critical time intervention: model
description and implications for the significance of timing in social
work interventions. Res Soc Work Pract. 2010;20(5):502–508. https://
doi.org/10.1177/1049731509360667.
28. Byrne T, Culhane D. Leveraging public healthcare dollars to fund
critical time intervention: a proposal for a scalable solution to crisis
homelessness in the United States. Int J Homeless. 2022;2(2):210–222.
https://doi.org/10.5206/ijoh.2022.1.13726.
29. Byrne T, Munley EA, Fargo JD, Montgomery AE, Culhane DP. New
perspectives on community-level determinants of homelessness. J
Urban Aff. 2013;35(5):607–625. https://doi.org/10.1111/j.1467-9906.
2012.00643.x.
June 2024
Downloaded for Anonymous User (n/a) at Harvard University from ClinicalKey.com by Elsevier on May 23, 2024.
For personal use only. No other uses without permission. Copyright ©2024. Elsevier Inc. All rights reserved.
1007
30. Grimes PW, Chressanthis GA. Assessing the effect of rent control on
homelessness. J Urban Econ. 1997;41(1):23–37. https://doi.org/10.
1006/juec.1996.1085.
31. Lee BA, Price-Spratlen T, Kanan JW. Determinants of homelessness
in metropolitan areas. J Urban Aff. 2003;25(3):335–356. https://doi.
org/10.1111/1467-9906.00168.
32. Nisar H, Vachon M, Horseman C, Murdoch J. Market predictors of
homelessness: how housing and community factors shape homeless-
ness rates within continuums of care. https://www.huduser.gov/por-
tal/publications/Market-Predictors-of-Homelessness.html. Accessed
May 19, 2023.
33. Rodriguez Luis A, Thomas Tainayah W, Finertie H, et al. Identifying
predictors of homelessness among adults in a large integrated health
system in northern California. Perm J. 2023;27(1):56–71. https://doi.
org/10.7812/TPP/22.096.
34. Cebula RJ, Alexander GM. Economic and noneconomic factors influ-
encing geographic differentials in homelessness: an exploratory state-
level analysis. Am J Econ Sociol. 2020;79(2):511–540. https://doi.org/
10.1111/ajes.12320.
35. Tsai J, Cao X. Association between suicide attempts and homelessness
in a population-based sample of U.S. Veterans and non-veterans. J
Epidemiology Community Health. 2019;73(4):346–352. https://doi.
org/10.1136/jech-2018-211065.
36. Hoffberg AS, Spitzer E, Mackelprang JL, Farro SA, Brenner LA. Sui-
cidal self-directed violence among homeless U.S. Veterans: a system-
atic review. Suicide Life-Threat Behav. 2018;48(4):481–498. https://
doi.org/10.1111/sltb.12369.