Stroke

CLINICAL AND POPULATION SCIENCES

Artificial Intelligence for Clinical Decision Support

in Acute Ischemic Stroke: A Systematic Review
Ela Marie Z. Akay *; Adam Hilbert , MSc*; Benjamin G. Carlisle , PhD; Vince I. Madai , MD, PhD; Matthias A. Mutke , MD;
Dietmar Frey , MD, JD

BACKGROUND: Established randomized trial-based parameters for acute ischemic stroke group patients into generic treatment
groups, leading to attempts using various artificial intelligence (AI) methods to directly correlate patient characteristics to
outcomes and thereby provide decision support to stroke clinicians. We review AI-based clinical decision support systems in
the development stage, specifically regarding methodological robustness and constraints for clinical implementation.

METHODS: Our systematic review included full-text English language publications proposing a clinical decision support system
using AI techniques for direct decision support in acute ischemic stroke cases in adult patients. We (1) describe data
and outcomes used in those systems, (2) estimate the systems’ benefits compared with traditional stroke diagnosis and
treatment, and (3) reported concordance with reporting standards for AI in healthcare.

RESULTS: One hundred twenty-one studies met our inclusion criteria. Sixty-five were included for full extraction. In our sample,
utilized data sources, methods, and reporting practices were highly heterogeneous.

CONCLUSIONS: Our results suggest significant validity threats, dissonance in reporting practices, and challenges to clinical
translation. We outline practical recommendations for the successful implementation of AI research in acute ischemic stroke
treatment and diagnosis.
Downloaded from http://ahajournals.org by on August 26, 2023

Key Words: artificial intelligence ◼ decision support ◼ humans ◼ intelligence ◼ ischemic stroke

I
schemic stroke is a leading cause of death and disability
worldwide and, without effective diagnostic and treat-
ment strategies, its burden is expected to increase.1 In
large prospective randomized trials, time from symptom
onset and a so-called imaging mismatch on perfusion
imaging have emerged as established parameters for
treatment decisions in acute ischemic stroke. However,
those parameters apply thresholds that are population-
based and not individualized.2 The rise of artificial intel-
ligence (AI) methods and their application in other areas
of medicine has inspired an attempt to use intelligent,
individualized, data-driven decision aids to improve this
aspect of stroke diagnosis and treatment.
See related article, p 1517
AI can use complex algorithms to provide decision aids
offering information and guidance to physicians extend-
ing beyond the traditional scoring systems.3 AI-based
clinical decision support systems (CDSSs) have already
been developed for the diagnosis of ischemic stroke and
are commercially available. Most of them aim to automate
subtasks such as the calculation of the Alberta Stroke
Program Early Computer Tomography (CT) Score4,5 or
the identification of ischemic lesion biomarkers (eg, per-
fusion maps) on imaging.6,7 However, AI-based solutions

 
Correspondence to:Ela Marie Zahide Akay, Charité Lab for Artificial Intelligence in Medicine (CLAIM), Charité Universitätsmedizin Berlin, 10117 Berlin, Germany.
Email ela-marie.akay@charite.de
*E.M.Z. Akay and A. Hilbert contributed equally.
Preprint posted on Research Square June 3, 2022. doi: https://doi.org/10.21203/rs.3.rs-1706474/v3.
Supplemental Material is available at https://www.ahajournals.org/doi/suppl/10.1161/STROKEAHA.122.041442.
For Sources of Funding and Disclosures, see page 1514
© 2023 The Authors. Stroke is published on behalf of the American Heart Association, Inc., by Wolters Kluwer Health, Inc. This is an open access article under the
terms of the Creative Commons Attribution License, which permits use, distribution, and reproduction in any medium, provided that the original work is properly cited.
Stroke is available at www.ahajournals.org/journal/str

Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442 June 2023   1505

 Akay et al
CLINICAL AND POPULATION
Nonstandard Abbreviations and Acronyms
SCIENCES
AI artificial intelligence
CDSS clinical decision support system
CT computer tomography
MINIMAR 
Minimum Information for Medical AI
Reporting
are not yet in use for direct decision support such as
treatment stratification or outcome prediction. This
review focuses on research regarding AI-based CDSSs
directly associating patient characteristics with clinical
outcomes as their development is a novel and promising
research field in acute ischemic stroke and could offer
great potential for both patients and clinicians.3 Because
of this, we do not discuss common AI-enabled conven-
tional medical tools such as e-ASPECTS.4
Figure 1 shows the steps in developing a CDSS. In
the development stage, an initial hypothesis is tested
using retrospective analysis of data. This can be achieved
using conventional methods such as scoring or mismatch
detection by human or AI. However, AI models can also
be used to provide high-dimensional data analysis to find
complex associations between variables as well as corre-
late patient characteristics with clinical outcomes, thereby
going beyond conventional clinical methods. Promising
Downloaded from http://ahajournals.org by on August 26, 2023
models are then prospectively validated similarly to exist-
ing biomarkers. In this review, we aim to provide an over-
view of AI-based CDSSs in the developmental stage to
determine which AI use cases, models, and paradigms
hold the potential to be turned into clinical tools. Fur-
thermore, we aim at identifying problems in the current
research landscape and providing recommendations for
future research.
Although existing reviews on AI-based solutions in
ischemic stroke are limited to certain data modalities,8,9
target only diagnosis and specific cases of stroke,10,11
or focus solely on interventions,12 this systematic review
analyzed the wide range of AI models proposed for
stroke diagnosis and treatment decision support, with-
out restrictions to populations, interventions or data
modalities. We hypothesize that there will be a wide vari-
ety of methods using AI with little standardization and
that the overall methodological robustness of method
validation is still to be improved upon to be viable in clini-
cal practice.
METHODS
This systematic review was preregistered on the Open Science
Framework. All data from this study is available on the Open
Science Framework page. For details see https://osf.io/
x5mb3/. Institutional approval or informed consent was not
required.
1506   June 2023
AI for Clinical Decision Support in Acute Stroke
Outcome Measures
The primary aim of this study was to generate an overview of
different decision support methods using AI for acute isch-
emic stroke focusing on associations to outcome measures.
Our secondary goals were to estimate the benefit of AI use
in stroke care in comparison to conventional decision-making,
assess which patients might profit most from AI-based deci-
sion-making, and assess the potential for integration into the
clinical workflow and the current methodological quality among
different approaches.
Literature Search, Inclusion Criteria
Trials were captured by searching Embase, Medline, ArXiv,
BiorXiv, MedrXiv, and Clinicaltrials.gov on May 17 and 19, 2021
for trials using the terms “stroke” and variations and combined
results with artificial intelligence and similar terms (full search
strategy available in Table S1).
Publications were screened by E.M.Z.A., A.H., and B.G.C.
for the following inclusion criteria: full-text publication, English
language, human research subjects, using an AI method, adult
patients with a confirmed diagnosis of acute ischemic stroke,
developing a method to be used in decision support in the acute
setting. We included only articles using an AI-based CDSS cor-
relating patient characteristics to clinical outcomes. Therefore,
automated stroke scoring and stroke segmentation methods
were excluded from this review.
Extraction
We devised an extraction template (see Supplemental Material)
that captured variables in the following domains: AI tech-
nique, patient characteristics, dataset specifications, validation
method, outcome end point, results, clinical comparator. Criteria
for extraction were prespecified in a codebook, and coders
underwent training before data collection. We performed data
extraction with Numbat Systematic Review Manager v. 2.13
(RRID:SCR_019207). Data from each article was extracted
by 2 researchers (E.M.Z.A. and A.H.) working independently.
In case of disagreements, the article was discussed with all
3 investigators (E.M.Z.A., A.H., and B.G.C.) until a unanimous
agreement was achieved.
Reporting Guidelines
As a post hoc analysis, we also report each trial’s concor-
dance with the Minimum Information for Medical AI Reporting
(MINIMAR) checklist.13 MINIMAR was designed to standardize
reporting on AI in medicine. The full list of criteria can be found
in Table 1. We specified criteria to clarify their representation
in our data. Criteria concerning the sourcing, preparation, and
availability of data as well as external validation, target user, and
the distinction between race and ethnicity were deemed out of
scope for this review. Cohort selection was defined as “patient
subgroups” as reported by the authors of a study. Data splitting
was defined as train/test splitting of the dataset and “internal
validation” was defined as the validation method. Gold standard
was defined as “clinical comparator,” meaning a method used
for the same classification task in a clinical setting, for example,
a clinical score or a human rating. Model task was defined as
decision support pertaining to the present (classification) or the
future (prediction).
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
 Akay et al
Downloaded from http://ahajournals.org by on August 26, 2023
Figure 1. Overview of steps in developing a clinical decision support system (CDSS).
ASPECTS indicates Alberta Stroke Program Early CT Score.
RESULTS
A total of 121 studies met our inclusion criteria. Among
these, 65 were included for full extraction whereas 20
studies were identified as proposing a method for auto-
mated stroke scoring, and 36 for stroke lesion segmenta-
tion in imaging. An overview of the articles proposing a
lesion segmentation method can be found in the Supple-
mental Material. Of the 20 articles on automated stroke
scoring, 18 were automated calculations of the Alberta
Stroke Program Early CT Score, and 2 reported methods
for automated collateral score calculation. Figure 2 shows
the Preferred Reporting Items for Systematic Reviews
and Meta-Analysis flowchart. Table 2 shows an overview
of all included studies and their results. The full table with
all extracted information can be found in Table S2.
Reporting Guidelines
A summary of the number of studies meeting specific
MINIMAR criteria can be found in Table 1. In general,
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
AI for Clinical Decision Support in Acute Stroke
CLINICAL AND POPULATION
SCIENCES
there was moderate to low adherence to the included
guidelines.
Patient Cohorts
All extracted studies provided information on the patient
population used. The median sample size in our sample
was 220 patients, with a range from 4 patients whose
imaging was processed voxel-wise35 to 2604 patients.14
50 (77%) of the studies specified stroke subgroups
(Table 2). The most common groups specified were
treatment groups including 20 (31%) with mechanical
thrombectomy and 9 (14%) with tissue-type plasmino-
gen activator. Nineteen (29%) of the studies included
only patients with anterior circulation stroke (anterior and
middle cerebral artery) and 7 (11%) with strokes within
the territory of the middle cerebral artery only.
We also extracted the demographic data reported.
Eight studies (12%) reported no demographic data. Most
studies included information on age (54/65, 83%) and
sex (52/65, 80%). Race was reported by 2 (3%) out
June 2023   1507
 Akay et al AI for Clinical Decision Support in Acute Stroke

Table 1.  MINIMAR Criteria and Number of Studies That technique whereas others relied on a high volume of
CLINICAL AND POPULATION

Meet Them clinical data points. The highest number of features was
Criterion Specification Studies meeting criterion used by Kappelhof et al23 with 63 clinical features in
SCIENCES

Study population and setting combination with CT imaging. The most common clinical
 Population No. of patients 65/65 100% features used were Age (32/65, 49%), National Insti-
 Study setting Out-of-scope … …
tutes of Health Stroke Scale score (31/65, 48%), and
Sex (23/65, 35%). Two articles (3%) gave no further
 Data source Out-of-scope … …
specification of what patient characteristics were used
 Cohort selection Stroke subgroups 50/65 77%
for the model.
Patient demographic 57/65 88%
characteristics
 Age, y 54/65 83% Model Technique
 Sex 52/65 80%
The reported target outcome end point for prediction can
 Race 2/65 3% be seen in Figure 3. Almost a third of the articles (21/65,
 Ethnicity Out-of-scope … … 32%) proposed methods for the dichotomized predic-
 Socioeconomic 0/65 … tion of the modified Rankin Scale score at 90 days after
status stroke. Final infarct prediction (17/65, 26%) and infarct
Model architecture core mapping (8/65, 12%) were the second most com-
 Model output 65/65 100% mon prediction tasks. Successful treatment by mechani-
 Target user Out-of-scope … … cal thrombectomy or thrombolysis was predicted by 2
 Data splitting 38/65 58% studies (3%) each. A clinical comparator was reported in
 Gold standard Clinical compara- 24/65 37% 24 studies (37%) of which 20 were automated compara-
tor tors and 4 compared the proposed method to a human
 Model task Classification/pre- --- … reading of the data. Most common techniques were con-
diction (future or volutional neural networks (17/65, 26%) and Random
present decision
support)
Forest algorithms (11/65, 17%). Data splitting into a
training set for initial model fitting and a test set for inde-
 Model architecture 64/65 98%
pendent evaluation was reported for slightly more than
 Features 65/65 100%
half of the studies extracted (38/65, 58%).
Downloaded from http://ahajournals.org by on August 26, 2023

 Missingness Out-of-scope … …
Model evaluation
 Optimization 17/65 26% Optimization, Validation, and Outcome
 Internal model valida- Validation method 61/65 94% Measures
tion
We found considerable variance and a lack of reporting
 External model Out-of-scope … …
validation
of standard machine learning methods. Internal model
validation was reported by 61 (94%) articles. The most
 Transparency (data Out-of-scope … …
availability) used performance measurement was the area under the
MINIMAR indicates Minimum Information for Medical Artificial Intelligence
curve, used in 54 (83%) of our samples. A clinical com-
Reporting. parator as a gold standard was reported by 24 (37%)
articles. The comparator was outperformed by the model
of 65 studies whereas socioeconomic status was not in 18 of 24 cases (75%), 2 of 24 (8%) reported a worse
reported by any studies. performance by the model, and in 4 of 24 (17%) a direct
comparison was not obviously determinable. Individual
results for each study as an overview can be found in
Data Used in AI Models Table 2 and Table S2. For details concerning optimization,
For the extraction of the data used as features for the AI validation, and outcome measures, see the Supplemental
model, we differentiated between magnetic resonance Material.
imaging, CT, other imaging, and clinical data. Twenty-
three (35%) of 65 studies used both image information
and clinical data for their proposed method. Raw imag- DISCUSSION
ing data was used by 22 (34%) studies for magnetic This systematic review gives an overview of the CDSSs
resonance imaging and 10 (15%) for CT, whereas 10 for acute ischemic stroke using AI technology. Our
(15%) studies relied only on clinical data. The clinical search revealed a high level of diversity in use cases,
data used varied greatly within the dataset of extracted methodologies, patient subgroups, and input data rep-
studies. Some used few clinical features with an imaging resenting the potentially wide-reaching benefits of AI in

1508   June 2023 Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442

 Akay et al
Downloaded from http://ahajournals.org by on August 26, 2023
stroke diagnosis and treatment. We examined AI-based
CDSS directly correlating patient characteristics to out-
comes, thereby going beyond the use of AI for improv-
ing conventional solutions such as automated Alberta
Stroke Program Early CT Score scoring or extraction
of perfusion maps. Although the studies included in
this review are focused on models in the early stages
of development, the performances achieved by these
approaches are promising and anticipate the success
of their future use in the clinical workflow. However,
to be translated into practical clinical tools, transpar-
ent, responsible, and validated research is essential.
This review revealed several pitfalls in the methodologi-
cal robustness of AI development for acute ischemic
stroke. In the following, we will describe these pitfalls
and offer recommendations to researchers and other
stakeholders involved in the translation of AI into clini-
cal practice.
Validity Threats
In the first step towards clinical translation, AI models
should be assessed by their performance metrics and
the methodological robustness of their development.
To allow such an assessment as well as a comparison
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
AI for Clinical Decision Support in Acute Stroke
CLINICAL AND POPULATION
SCIENCES
Figure 2. Preferred Reporting Items
for Systematic Reviews and Meta-
Analysis flowchart.
among different models, standardized reporting is
necessary.
First, we saw limited reporting on the measures taken
by the authors to guarantee the technical robustness
of a model. This is usually achieved by simulating real-
world scenarios of varying quality, distribution, and noise
in datasets. The most important step in this process is to
split the available data into different subsets: A training
set is used to fit the parameters of the model to the task
at hand. The validation set is used for model selection and
optimization whereas the test set is held out throughout
the model optimization process and only used to evalu-
ate the optimized model’s performance. Almost half of
the reviewed articles did not describe the use of an
additional test set, thus presumably reporting validation
performances. Validation results are naturally overfitted
and often skewed to the given parameter combination
and optimization process. This means that even though
these parameters deliver the best model, performance
measured on the validation set does not necessarily
reflect performance in a real-world setting. Additionally,
these datasets need to be strictly separated from each
other, meaning that even if the use of a test set was
reported, there is a risk of data leakage between datas-
ets, further limiting the validity of a given approach.79 This
June 2023   1509
 Akay et al AI for Clinical Decision Support in Acute Stroke

Table 2.  Overview of Studies

CLINICAL AND POPULATION

Data
References Outcome end point Patient subgroup Clinical Imaging splitting AUC Dice
SCIENCES

90-d mRS
 Heo et al14 90-d mRS No recanalization 38 × × 0.888 ×
therapy (ns)
 Seiffge et al15 90-d mRS tPA 5 × ✓ 0.786 ×
 Monteiro et al 16
90-d mRS tPA 49 x × 0.808 ×
 Chung et al17 90-d mRS, Successful Elderly, <3 h 9 × × 0.974 ×
thrombolysis
 Alaka et al18 90-d mRS ns 14 × ✓ 0.71 ×
 Zihni et al19 90-d mRS Supratentorial 7 × ✓ 0.83 ×
 Hilbert et al20 90-d mRS, successful Thrombectomy × CT × mTICI, 0.65; mRS, ×
recanalization 0.71
 Nishi et al21 90-d mRS Thrombectomy, anterior × MRI ✓ 0.73 ×
 Heo et al22 90-d mRS First time × MRI ✓ 0.805 ×
(report)
 Kappelhof et al23 90-d mRS, mRS after Thrombectomy 63 CT × × ×
recanalization
 van Os et al24 90-d mRS, successful Thrombectomy, anterior 27 CT ✓ mTICI, 0.55; mRS, ×
recanalization 0.79
 Xie et al25 90-d mRS Anterior 3 CT × 0.748 ×
 Alawieh et al26 90-d mRS Thrombectomy, elderly 9 CT ✓ × ×
 Bacchi et al27 90-d mRS, NIHSS, 24 h ns ns CT ✓ mRS, 0.75; NIHSS, mRS, 0.69;
0.70 NIHSS, 0.74
 Brugnara et al28 90-d mRS Thrombectomy, anterior 13 CT × 0.747 ×
 Ramos et al29 90-d mRS Thrombectomy, anterior 32 CT ✓ 0.81 ×
 Nishi et al 30
90-d mRS Thrombectomy, anterior 15 CT ✓ 0.90 ×
Downloaded from http://ahajournals.org by on August 26, 2023

 Chung et al31 90-d mRS, ≥8 point tPA, age 18–80; 10 MRI/CT ✓ × ×

NIHSS improvement at NIHSS, 4–25
24 h
 Matsumoto et al32 90-d mRS, in-hospital ns 49 MRI/CT ✓ mRS, 0.92; Mort, 0.84 ×
mortality
 Hamann et al33 90-d mRS MCA, thrombectomy, 12 MRI ✓ 0.684 ×
M1, <6 h
 Jiang et al34 90-d mRS Anterior 3 CT × 0.80 ×
Final infarct prediction
 Giacalone et al35 Final infarct Anterior × MRI × × ×
 Bagher-Ebadian Final infarct ns × MRI × 0.89 ×
et al36
 McKinley et al37 Final infarct with or without MCA × MRI ✓ Positive, 0.94; nega- ×
reperfusion tive, 0.96
 Livne et al38 Final infarct ns × MRI × 0.92 ×
 Nielsen et al39 Final infarct tPA × MRI ✓ 0.88 ×
 Lucas et al 40
Final infarct Thrombectomy × CT ✓ × 0.43
 Tozlu et al41 Final infarct Anterior, <6/12 h, tPA/ × MRI ✓ 0.88 0.28
conservative treatment
 Ho42 Final infarct MCA × MRI × 0.871 0.347
 Qiu et al 43
Final infarct ns × CT ✓ × 0.447
 Yu et al44 Final infarct Thrombectomy, anterior × MRI ✓ 0.92 0.53
 Grosser et al45 Final infarct First time, anterior, × MRI × 0.893 0.387
NIHSS >4, <12 h
 Kim et al46 Final infarct with and MCA, thrombectomy × MRI ✓ × 0.49
without recanalization
 Grosser et al47 Final infarct Anterior, <12 h × MRI × 0.872 0.348
(Continued )

1510   June 2023 Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442

 Akay et al AI for Clinical Decision Support in Acute Stroke

Table 2. Continued

CLINICAL AND POPULATION

Data
References Outcome end point Patient subgroup Clinical Imaging splitting AUC Dice

SCIENCES
 Pinto et al48 Final infarct Thrombectomy × MRI ✓ × 0.38
 Debs et al 49
Final infarct Thrombectomy × MRI × Reperf, 0.87; nonre- Reperf, 0.43;
perf, 0.81 nonreperf, 0.44
 Winder et al50 Final infarct Thrombectomy, tPA, 4 MRI × × 0.464
distal ICA/M1
 Robben et al51 Final infarct Thrombectomy, anterior, 1 CT × 0.54 (PR) 0.47
≤6 h
Infarct core mapping
 Sheth et al52 Infarct core, LVO Anterior × CT × 30 mL, 0.88; 50 mL, ×
0.90; LVO, 0.844
 Rava et al53 Infarct core Thrombectomy, success- × DSA ✓ 0.904 ×
ful recanalization
 Hakim et al54 Infarct core Thrombectomy, anterior × CT ✓ × 0.51
 Yu et al55 TaR, infarct core ns × MRI ✓ TaR, 0.92; core, 0.94 TaR, 0.60; core,
0.57
 Kasasbeh et al56 Infarct core Anterior, <8 h 4 CT ✓ 0.87 0.43
Intracranial hemorrhage
 Chung et al57 sICH, 90-d mortality tPA 5/6 × × siCH, 0.941; Mort, ×
0.976
 Wang et al58 sICH tPA 5 × ✓ 0.82 ×
 Yu et al59 sICH <6 h × MRI × 0.837 0.717
 Dharmasaroja and sICH tPA 27 CT ✓ 0.788 0.522
Dharmasaroja60
 Bentley et al61 sICH tPA 1 CT × 0.744 ×
LVO
 Smith et al62 LVO ns × Head- × 0.79 ×
Downloaded from http://ahajournals.org by on August 26, 2023

pulse
 Stib et al63 LVO ns × CT ✓ 0.89 ×
 Meijs et al 64
LVO Anterior × CT ✓ 0.98 ×
 You et al65 LVO ns ns CT ✓ 0.847 0.804
Onset time
 Ho et al66 Onset time MCA × MRI × 0.683 0.765
 Ho et al67 Onset time MCA × MRI ✓ 0.765 0.788
 Lee et al68 Onset time ns × MRI × 0.851 ×
 Zhang et al 69
Onset time ns × MRI ✓ 0.74 ×
Other
 Ho et al70 Discharge mortality ns 6 × ✓ × 0.5
 Li et al71 Poststroke pneumonia ns 6 × ✓ 0.841 ×
 Li et al72
6-mo mRS ns 21 × ✓ 0.874 ×
 Qiu et al73 Successful thrombolysis tPA, ICA/M1 MCA × CT × 0.85 ×
 Wang et al74 Penumbral tissue Thrombectomy, anterior × MRI ✓ 0.959 0.47
 Hofmeister et al 75
First-time recanalization, Thrombectomy × CT ✓ 1st pass, 0.88 ×
number of passages
 Fu et al76 Edema MCA 3 CT × 0.96 0.91
 Sung et al 77
Worsening of NIHSS NIHSS≥3 17 MRI/CT × 0.934 0.8
within 3 d
 Velagapudi et al78 First-time recanalization Thrombectomy 20 MRI/CT ✓ 0.659 ×
AUC indicates area under the curve; CT, computer tomography; LVO, large vessel occlusion; MCA, middle cerebral artery occlusion; Mort, mortality; MRI, magnetic
resonance imaging; mRS, modified Rankin Scale score; mTICI, modified Thrombolysis in Cerebral Infarction score; NIHSS, National Institutes of Health Stroke Scale
score; PR, precision-recall; sICH, symptomatic intracranial hemorrhage; TaR, tissue at risk; and tPA, tissue-type plasminogen activator.

Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442 June 2023   1511

 Akay et al
CLINICAL AND POPULATION
SCIENCES
Figure 3. Outcome end points across studies.
mRS, modified Rankin Scale score; NIHSS, National Institutes of Health Stroke Scale score; and tPA, tissue-type plasminogen activator.
implies that the majority of the AI models proposed for
stroke CDSSs to date would need additional, rigorous
testing and external validation to assess applicability in
clinical practice. Another important step towards model
robustness is the use of an efficient search mechanism
to find the best parameters across a well-defined param-
eter space. This process, called hyper-parameter tuning
ensures that the model architecture is utilized to yield
Downloaded from http://ahajournals.org by on August 26, 2023
optimal performance. Although model architecture is
well-reported, only a quarter of the works provide detail
on hyper-parameter tuning practices.
Second, our analysis shows a systematically insuf-
ficient description of patient demographic information
beyond variables used by the developed model. Only a
marginal percentage of the studies report race or socio-
economic status. This represents a major risk of bias as
it limits generalizability and at worst disadvantages cer-
tain demographic groups when applied in a real-world
scenario.
Beyond the MINIMAR criteria, we found a large
degree of heterogeneity in reporting practices in the
reviewed articles, making comparison between study
outcomes difficult. The most used performance mea-
surement was the area under the curve and whereas it is
a robust metric, reporting sensitivity, specificity, balanced
accuracy and a calibration metric (eg, Brier score) facili-
tates a more reliable interpretation. We saw minor align-
ment across the reviewed studies with many of them
failing to report metrics that allow for obvious interpreta-
tion and comparison to other methods.
Overall, the reporting standards among different
articles do not allow for an effective assessment of the
proposed methods. This further impedes the develop-
ment of models that are reproducible as well as gen-
eralizable to datasets with different characteristics.
1512   June 2023
AI for Clinical Decision Support in Acute Stroke
Although poor reporting standardization is one of the
main factors limiting reproducibility, another is the lack
of openly available datasets for researchers to use. The
dependence of AI on sufficient labeled data to yield
models with proper generalization and reliable results
has been discussed frequently.80 Due to data privacy
concerns, this is a challenge for all medical AI devel-
opment. On a positive note, the rise in use of exten-
sive data collected by national stroke registries holds
great promise as these and other comprehensive data-
bases not only enable model development on more
data points but also provide a close sample to general
stroke populations.
Clinical Translation
To fully exploit the potential impact of AI-based CDSSs,
clinical feasibility must be prioritized, and efforts must
be aimed at solving challenges of the clinical workflow.
This requires researchers to design practical models and
clearly delineate the requirements and added value of a
given model within the fast-paced clinical workflow. Tools
using data or variables that are not routinely acquired will
impede efficiency as will models that rely on an extensive
set of clinical variables or imaging.
To facilitate clinical translation the input values for
an AI model need to be quickly available and routinely
acquired. However, our analysis showed that close to
half of the articles involving clinical parameters employed
more than 10 variables and almost a quarter of articles
using imaging relied on the acquisition of at least 3
sequences. Even though major centers routinely image
several sequences, for example, CT, CT angiography, and
CT perfusion, AI should prioritize data-efficient solutions
to increase the accessibility and speed of the proposed
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
 Akay et al
models. Moreover, we saw that the lack of reporting stan-
dards also leads to studies that do not clearly delineate
what input data is needed. This goes so far as some stud-
ies reporting the use of clinical data but not describing
what clinical features were actually used by the model
(Table 2).
When discussing the added value of a CDSS within
the clinical workflow, our results suggest that the defini-
tion of decision support in stroke literature is ambigu-
ous. We identified 3 main categories: (1) models for
automatically extracting stroke scores, which were not
included in this review since they do not augment the
range of available information for decision-making; (2)
models classifying patients for easier treatment selec-
tion; and (3) models for predicting a future outcome.
Both the second and third categories can aid in patient
stratification by providing complementary informa-
tion with regard to the likely benefit of treatment. Of
particular note is that only 6% of articles proposed a
model directly predicting the success of mechanical
thrombectomy, even though improving the specificity of
patient stratification is one of the main aims of stroke
treatment research.
One of the unique benefits offered by AI solutions
is the extraction of as-yet-unknown predictors of out-
comes from raw imaging. However, only 2 models used
imaging for functional outcome prediction. Considering
the predictive performance of clinical data and estab-
lished biomarkers remain superior, the exploitation of AI-
Downloaded from http://ahajournals.org by on August 26, 2023
driven image processing for outcome prognosis is rather
limited.
Implications and Recommendations
First, we caution researchers to better adhere to best
practices in model development such as data splitting
and hyper-parameter tuning. Evaluation of model perfor-
mance on hold-out test sets not involved in model train-
ing, model selection, and hyper-parameter tuning should
be warranted. Modern hyper-parameter tuning such as
Bayesian optimization, hyperband, or spectral analysis
approaches should be evaluated for the given use case81
and when applicable, favored over classical grid or ran-
dom search methods. Researchers should involve clinical
practitioners in the design process from the early stages
of model development to ensure feasibility in daily clini-
cal practice. Researchers must also review, implement
and report on relevant trustworthy and reliability consid-
erations such as technical robustness and transparency
prescribed by, for example, the EU Ethics Guidelines for
Trustworthy AI.82
Second, we call for journal editors and reviewers to
demand adherence to stricter reporting requirements,
for example, MINIMAR. Describing a targeted use case
and clinical decision to aid in publications of such work
is also of great importance. Here, we recommend that
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
AI for Clinical Decision Support in Acute Stroke
researchers and reviewers in the field use diagnosis
CLINICAL AND POPULATION
and treatment guidelines provided by regional stroke
organizations.
SCIENCES
Third, we encourage funders to prioritize projects that
focus on decision support tools with a clear outline of fea-
sible integration into real-life clinical care, including angles
of trustworthiness, usability, technical robustness, and
data governance.83 Specific considerations of stroke care
such as acute, timely predictions, inclusion of neuroim-
aging, and an absolute user-centered approach to effec-
tively unburden medical professionals should be taken
into account and evaluated by a multidisciplinary team of
AI scientists and engineers, clinicians, and ethicists.
Finally, we would like to highlight the crucial role of
high-quality training and validation data. Here, policy-
makers have a key role to provide a path for research-
ers to obtain the necessary plurality of data and to meet
the requirements of robust model development and
validation.
Limitations
Our study has several limitations. First, our study included
only published research articles, making it susceptible to
publication bias. Even though unpublished works might
shed light on further, novel methods for supporting deci-
sions, practical implementation of AI algorithms in clinical
settings must rely on rigorously peer-reviewed solutions.
Hence, we do not see unpublished works having an
influential effect on the state-of-the-art of AI in stroke
decision support. Second, this study was descriptive in
nature, where we elaborated on trends and presented the
distribution of contributions in the field from some spe-
cific angles. However, we did not formally test the trans-
lated impact and did not carry out targeted quantitative
analyses to corroborate our claims. Since the MINIMAR
analysis was performed post hoc, some aspects of the
checklist were not included in this review although they
might be of interest to readers and researchers. All the
highlighted shortcomings and derived recommendations
are based on theoretical interpretations and thus do not
reflect the actual impact of practical implementations.
Conclusions
Although there have been great advances in grow-
ing availability of research data to promote medical AI
development, the research on clinical decision-making in
stroke is still limited. Within the existing literature, better
reporting of AI techniques applied to the context of acute
ischemic stroke care is required and best practices of AI
model development should be adopted by researchers.
If correctly implemented, these approaches make use
of individualized patient data while providing additional
information to the physician could lead to better patient
outcomes in acute ischemic stroke.
June 2023   1513
 Akay et al
ARTICLE INFORMATION
CLINICAL AND POPULATION
Received September 26, 2022; final revision received January 10, 2023; ac-
cepted February 21, 2023.
SCIENCES
Affiliations
Charité Lab for Artificial Intelligence in Medicine (CLAIM) (E.M.Z.A., A.H., D.F.) and
QUEST Center for Responsible Research, Berlin Institute of Health (BIH) (B.G.C.,
V.I.M.), Charité Universitätsmedizin Berlin, Germany. Faculty of Computing, Engi-
neering and the Built Environment, School of Computing and Digital Technology,
Birmingham City University, United Kingdom (V.I.M.). Department of Neuroradiol-
ogy, Heidelberg University Hospital, Germany (M.A.M.).
Sources of Funding
This work has received funding from the German Federal Ministry of Education
and Research through a GOBio grant for the research group PREDICTioN2020
(lead: Dr Frey), and funding from the European Commission via the Horizon 2020
program for Precise4Q (No. 777107, lead: Dr Frey).
Disclosures
A. Hilbert and Dr Madai reported receiving personal fees from ai4medicine out-
side the submitted work. Dr Frey reported receiving grants from the European
Commission, reported receiving personal fees from and holding an equity interest
in ai4medicine outside the submitted work. The other authors report no conflicts.
Supplemental Material
PRISMA Checklist
PRISMA Abstract Checklist
Tables S1–S3
Methods
REFERENCES
1. GBD 2015 Neurological Disorders Collaborator Group. Global, regional,
and national burden of neurological disorders during 1990-2015: a system-
atic analysis for the Global Burden of Disease Study 2015. Lancet Neurol.
2017;16:877–897. doi: 10.1016/S1474-4422(17)30299-5
Downloaded from http://ahajournals.org by on August 26, 2023
2. Li KHC, Jesuthasan A, Kui C, Davies R, Tse G, Lip GYH. Acute ischemic
stroke management: concepts and controversies.A narrative review. Expert
Rev Neurother. 2021;21:65–79. doi: 10.1080/14737175.2021.1836963
3. Jamieson T, Goldfarb A. Clinical considerations when applying machine
learning to decision-support tasks versus automation. BMJ Qual Saf.
2019;28:778–781. doi: 10.1136/bmjqs-2019-009514
4. Nagel S, Sinha D, Day D, Reith W, Chapot R, Papanagiotou P, Warburton EA,
Guyler P, Tysoe S, Fassbender K, et al. e-ASPECTS software is non-inferior
to neuroradiologists in applying the ASPECT score to computed tomogra-
phy scans of acute ischemic stroke patients. Int J Stroke. 2017;12:615–
622. doi: 10.1177/1747493016681020
5. Pfaff J, Herweh C, Schieber S, Schönenberger S, Bösel J, Ringleb PA,
Möhlenbruch M, Bendszus M, Nagel S. e-ASPECTS Correlates with and Is
Predictive of Outcome after Mechanical Thrombectomy. AJNR Am J Neuro-
radiol. 2017;38:1594–1599. doi: 10.3174/ajnr.A5236
6. Grunwald IQ, Kulikovski J, Reith W, Gerry S, Namias R, Politi M,
Papanagiotou P, Essig M, Mathur S, Joly O, et al. Collateral automation for
triage in stroke: evaluating automated scoring of collaterals in acute stroke
on computed tomography scans. Cerebrovasc Dis. 2019;47:217–222. doi:
10.1159/000500076
7. Kellner E, Reisert M, Kiselev VG, Maurer CJ, Beume LA, Urbach H, Egger K.
Automated infarct core volumetry within the hypoperfused tissue: technical
implementation and evaluation. J Comput Assist Tomogr. 2017;41:515–
520. doi: 10.1097/RCT.0000000000000570
8. Bivard A, Churilov L, Parsons M. Artificial intelligence for decision support in
acute stroke — current roles and potential. Nat Rev Neurol. 2020;16:575–
585. doi: 10.1038/s41582-020-0390-y
9. Soun JE, Chow DS, Nagamine M, Takhtawala RS, Filippi CG, Yu W,
Chang PD. Artificial intelligence and acute stroke imaging. Am J Neuroradiol.
2021;42:2–11. doi: 10.3174/ajnr.A6883
10. Karthik R, Menaka R, Johnson A, Anand S. Neuroimaging and deep learn-
ing for brain stroke detection - A review of recent advancements and future
prospects. Comput Methods Programs Biomed. 2020;197:105728. doi:
10.1016/j.cmpb.2020.105728
11. Murray NM, Unberath M, Hager GD, Hui FK. Artificial intelligence
to diagnose ischemic stroke and identify large vessel occlusions: a
1514   June 2023
AI for Clinical Decision Support in Acute Stroke
systematic review. J NeuroInterventional Surg. 2020;12:156–164. doi:
10.1136/neurintsurg-2019-015135
12. Lotan E. Emerging artificial intelligence imaging applications for stroke
interventions. AJNR Am J Neuroradiol. 2021;42:255–256.
13. Hernandez-Boussard T, Bozkurt S, Ioannidis JPA, Shah NH. MINIMAR
(MINimum Information for Medical AI Reporting): developing reporting
standards for artificial intelligence in health care. J Am Med Inform Assoc.
2020;27:2011–2015. doi: 10.1093/jamia/ocaa088
14. Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine learning–based
model for prediction of outcomes in acute stroke. Stroke. 2019;50:1263–
1265. doi: 10.1161/STROKEAHA.118.024293
15. Seiffge DJ, Karagiannis A, Strbian D, Gensicke H, Peters N, Bonati LH,
Kotisaari K, Leppä M, Kejda-Scharler J, Tränka C, et al. Simple variables
predict miserable outcome after intravenous thrombolysis. Eur J Neurol.
2014;21:185–191. doi: 10.1111/ene.12254
16. Monteiro M, Fonseca AC, Freitas AT, Pinho e Melo T, Francisco AP, Ferro JM,
Oliveira AL. Using machine learning to improve the prediction of functional
outcome in ischemic stroke patients. IEEE/ACM Trans Comput Biol Bioin-
form. 2018;15:1953–1959. doi: 10.1109/TCBB.2018.2811471
17. Chung CC, Chen YC, Hong CT, Chi NF, Hu CJ, Hu HH, Chan L,
Chiu HW. Artificial neural network-based analysis of the safety and efficacy
of thrombolysis for ischemic stroke in older adults in Taiwan. Neurol Asia.
2020;25:109–117.
18. Alaka SA, Menon BK, Brobbey A, Williamson T, Goyal M, Demchuk AM,
Hill MD, Sajobi TT. Functional outcome prediction in ischemic stroke: a com-
parison of machine learning algorithms and regression models. Front Neurol.
2020;11:889. doi: 10.3389/fneur.2020.00889
19. Zihni E, Madai VI, Livne M, Galinovic I, Khalil AA, Fiebach JB, Frey D. Open-
ing the black box of artificial intelligence for clinical decision support: a
study predicting stroke outcome. PLoS One. 2020;15:e0231166. doi:
10.1371/journal.pone.0231166
20. Hilbert A, Ramos LA, van Os HJA, Olabarriaga SD, Tolhuisen ML, Wermer MJH,
Barros RS, van der Schaaf I, Dippel D, Roos YBWEM, et al. Data-efficient
deep learning of radiological image data for outcome prediction after endo-
vascular treatment of patients with acute ischemic stroke. Comput Biol Med.
2019;115:103516. doi: 10.1016/j.compbiomed.2019.103516
21. Nishi H, Oishi N, Ishii A, Ono I, Ogura T, Sunohara T, Chihara H, Fukumitsu R,
Okawa M, Yamana N, et al. Deep learning–derived high-level neuroimag-
ing features predict clinical outcomes for large vessel occlusion. Stroke.
2020;51:1484–1492. doi: 10.1161/STROKEAHA.119.028101
22. Heo TS, Kim YS, Choi JM, Jeong YS, Seo SY, Lee JH, Jeon JP, Kim C.
Prediction of stroke outcome using natural language processing-based
machine learning of radiology report of brain MRI. J Pers Med. 2020;10:286.
doi: 10.3390/jpm10040286
23. Kappelhof N, Ramos LA, Kappelhof M, van Os HJA, Chalos V,
van Kranendonk KR, Kruyt ND, Roos YBWEM, van Zwam WH, van der Schaaf IC,
et al. Evolutionary algorithms and decision trees for predicting poor outcome
after endovascular treatment for acute ischemic stroke. Comput Biol Med.
2021;133:104414. doi: 10.1016/j.compbiomed.2021.104414
24. van Os HJA, Ramos LA, Hilbert A, van Leeuwen M, van Walderveen MAA,
Kruyt ND, Dippel DWJ, Steyerberg EW, van der Schaaf IC, Lingsma HF, et
al; MR CLEAN Registry Investigators. Predicting outcome of endovascular
treatment for acute ischemic stroke: potential value of machine learning
algorithms. Front Neurol. 2018;9:784. doi: 10.3389/fneur.2018.00784
25. Xie Y, Jiang B, Gong E, Li Y, Zhu G, Michel P, Wintermark M, Zaharchuk G. Use
of gradient boosting machine learning to predict patient outcome in acute
ischemic stroke on the basis of imaging, demographic, and clinical informa-
tion. Am J Roentgenol. 2019;212:44–51. doi: 10.2214/AJR.18.20260
26. Alawieh A, Zaraket F, Alawieh MB, Chatterjee AR, Spiotta A. Using
machine learning to optimize selection of elderly patients for endovas-
cular thrombectomy. J NeuroInterventional Surg. 2019;11:847–851. doi:
10.1136/neurintsurg-2018-014381
27. Bacchi S, Zerner T, Oakden-Rayner L, Kleinig T, Patel S, Jannes J. Deep
learning in the prediction of ischaemic stroke thrombolysis functional out-
comes. Acad Radiol. 2020;27:e19–e23. doi: 10.1016/j.acra.2019.03.015
28. Brugnara G, Neuberger U, Mahmutoglu MA, Foltyn M, Herweh C, Nagel S,
Schönenberger S, Heiland S, Ulfert C, Ringleb PA, et al. Multimodal pre-
dictive modeling of endovascular treatment outcome for acute isch-
emic stroke using machine-learning. Stroke. 2020;51:3541–3551. doi:
10.1161/STROKEAHA.120.030287
29. Ramos LA, Kappelhof M, van Os HJA, Chalos V, Van Kranendonk K, Kruyt ND,
Roos YBWEM, van der Lugt A, van Zwam WH, van der Schaaf IC, et al. Predicting
poor outcome before endovascular treatment in patients with acute ischemic
stroke. Front Neurol. 2020;11:580957. doi: 10.3389/fneur.2020.580957
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
 Akay et al
30. Nishi H, Oishi N, Ishii A, Ono I, Ogura T, Sunohara T, Chihara H, Fukumitsu R,
Okawa M, Yamana N, et al. Predicting clinical outcomes of large vessel
occlusion before mechanical thrombectomy using machine learning. Stroke.
2019;50:2379–2388. doi: 10.1161/STROKEAHA.119.025411
31. Chung CC, Hong CT, Huang YH, Su ECY, Chan L, Hu CJ, Chiu HW. Pre-
dicting major neurologic improvement and long-term outcome after throm-
bolysis using artificial neural networks. J Neurol Sci. 2020;410:116667. doi:
10.1016/j.jns.2020.116667
32. Matsumoto K, Nohara Y, Soejima H, Yonehara T, Nakashima N,
Kamouchi M. Stroke prognostic scores and data-driven prediction of clinical
outcomes after acute ischemic stroke. Stroke. 2020;51:1477–1483. doi:
10.1161/STROKEAHA.119.027300
33. Hamann J, Herzog L, Wehrli C, Dobrocky T, Bink A, Piccirelli M, Panos L,
Kaesmacher J, Fischer U, Stippich C, et al. Machine‐learning‐based outcome
prediction in stroke patients with middle cerebral artery‐M1 occlusions and early
thrombectomy. Eur J Neurol. 2021;28:1234–1243. doi: 10.1111/ene.14651
34. Jiang B, Zhu G, Xie Y, Heit JJ, Chen H, Li Y, Ding V, Eskandari A, Michel P,
Zaharchuk G. Prediction of clinical outcome in patients with large-vessel
acute ischemic stroke: performance of machine learning versus SPAN-100.
Am J Neuroradiol. 2021;42:240–246. doi: 10.3174/ajnr.A6918
35. Giacalone M, Rasti P, Debs N, Frindel C, Cho TH, Grenier E,
Rousseau D. Local spatio-temporal encoding of raw perfusion MRI for the
prediction of final lesion in stroke. Med Image Anal. 2018;50:117–126. doi:
10.1016/j.media.2018.08.008
36. Bagher-Ebadian H, Jafari-Khouzani K, Mitsias PD, Lu M, Soltanian-Zadeh H,
Chopp M, Ewing JR. Predicting final extent of ischemic infarction using
artificial neural network analysis of multi-parametric MRI in patients with
stroke. PLoS One. 2011;6:e22626. doi: 10.1371/journal.pone.0022626
37. McKinley R, Häni L, Gralla J, El-Koussy M, Bauer S, Arnold M, Fischer U,
Jung S, Mattmann K, Reyes M, et al. Fully automated stroke tissue estima-
tion using random forest classifiers (FASTER). J Cereb Blood Flow Metab.
2017;37:2728–2741. doi: 10.1177/0271678X16674221
38. Livne M, Boldsen JK, Mikkelsen IK, Fiebach JB, Sobesky J,
Mouridsen K. Boosted tree model reforms multimodal magnetic resonance
imaging infarct prediction in acute stroke. Stroke. 2018;49:912–918. doi:
10.1161/STROKEAHA.117.019440
39. Nielsen A, Hansen MB, Tietze A, Mouridsen K. Prediction of tis-
sue outcome and assessment of treatment effect in acute isch-
Downloaded from http://ahajournals.org by on August 26, 2023
emic stroke using deep learning. Stroke. 2018;49:1394–1401. doi:
10.1161/STROKEAHA.117.019740
40. Lucas C, Kemmling A, Bouteldja N, Aulmann LF, Madany Mamlouk A,
Heinrich MP. Learning to predict ischemic stroke growth on acute CT per-
fusion data by interpolating low-dimensional shape representations. Front
Neurol. 2018;9:989. doi: 10.3389/fneur.2018.00989
41. Tozlu C, Ozenne B, Cho TH, Nighoghossian N, Mikkelsen IK, Derex L,
Hermier M, Pedraza S, Fiehler J, Østergaard L, et al. Comparison of clas-
sification methods for tissue outcome after ischemic stroke. Eur J Neurosci.
2019;50:3590–3598.
42. Ho KC. Predicting ischemic stroke tissue fate using a deep convolutional
neural network on source magnetic resonance perfusion images. J Med
Imaging. 2019;6:1. doi: 10.1117/1.jmi.6.2.026001
43. Qiu W, Kuang H, Teleg E, Ospel JM, Sohn SI, Almekhlafi M, Goyal M, Hill MD,
Demchuk AM, Menon BK. Machine learning for detecting early infarction in
acute stroke with non–contrast-enhanced CT. Radiology. 2020;294:638–
644. doi: 10.1148/radiol.2020191193
44. Yu Y, Xie Y, Thamm T, Gong E, Ouyang J, Huang C, Christensen S, Marks MP,
Lansberg MG, Albers GW, et al. Use of deep learning to predict final isch-
emic stroke lesions from initial magnetic resonance imaging. JAMA Netw
Open. 2020;3:e200772. doi: 10.1001/jamanetworkopen.2020.0772
45. Grosser M, Gellißen S, Borchert P, Sedlacik J, Nawabi J, Fiehler J, Forkert ND.
Improved multi-parametric prediction of tissue outcome in acute ischemic
stroke patients using spatial features. PLoS One. 2020;15:e0228113. doi:
10.1371/journal.pone.0228113
46. Kim YC, Kim HJ, Chung JW, Kim IG, Seong MJ, Kim KH, Jeon P, Nam HS,
Seo WK, Kim GM, et al. Novel estimation of penumbra zone based on infarct
growth using machine learning techniques in acute ischemic stroke. J Clin
Med. 2020;9:1977. doi: 10.3390/jcm9061977
47. Grosser M, Gellißen S, Borchert P, Sedlacik J, Nawabi J, Fiehler J,
Forkert ND. Localized prediction of tissue outcome in acute ischemic stroke
patients using diffusion- and perfusion-weighted MRI datasets. PLoS One.
2020;15:e0241917. doi: 10.1371/journal.pone.0241917
48. Pinto A, Pereira S, Meier R, Wiest R, Alves V, Reyes M, Silva CA. Combining
unsupervised and supervised learning for predicting the final stroke lesion.
Med Image Anal. 2021;69:101888. doi: 10.1016/j.media.2020.101888
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442
AI for Clinical Decision Support in Acute Stroke
49. Debs N, Cho TH, Rousseau D, Berthezène Y, Buisson M, Eker O,
CLINICAL AND POPULATION
Mechtouff L, Nighoghossian N, Ovize M, Frindel C. Impact of the reperfusion
status for predicting the final stroke infarct using deep learning. NeuroImage
Clin. 2021;29:102548. doi: 10.1016/j.nicl.2020.102548
SCIENCES
50. Winder AJ, Siemonsen S, Flottmann F, Thomalla G, Fiehler J, Forkert ND.
Technical considerations of multi-parametric tissue outcome prediction
methods in acute ischemic stroke patients. Sci Rep. 2019;9:13208. doi:
10.1038/s41598-019-49460-y
51. Robben D, Boers AMM, Marquering HA, Langezaal LLCM, Roos YBWEM,
van Oostenbrugge RJ, van Zwam WH, Dippel DWJ, Majoie CBLM,
van der Lugt A, et al. Prediction of final infarct volume from native CT per-
fusion and treatment parameters using deep learning. Med Image Anal.
2020;59:101589. doi: 10.1016/j.media.2019.101589
52. Sheth SA, Lopez-Rivera V, Barman A, Grotta JC, Yoo AJ, Lee S, Inam ME,
Savitz SI, Giancardo L. Machine learning–enabled automated determination
of acute ischemic core from computed tomography angiography. Stroke.
2019;50:3093–3100. doi: 10.1161/STROKEAHA.119.026189
53. Rava RA, Mokin M, Snyder KV, Waqas M, Siddiqui AH, Davies JM, Levy EI,
Ionita CN. Performance of angiographic parametric imaging in locating
infarct core in large vessel occlusion acute ischemic stroke patients. J Med
Imaging. 2020;7:016001. doi: 10.1117/1.JMI.7.1.016001
54. Hakim A, Christensen S, Winzeck S, Lansberg MG, Parsons MW, Lucas C,
Robben D, Wiest R, Reyes M, Zaharchuk G. Predicting infarct core from
computed tomography perfusion in acute ischemia with machine learn-
ing: lessons from the ISLES challenge. Stroke. 2021;52:2328–2337. doi:
10.1161/STROKEAHA.120.030696
55. Yu Y, Xie Y, Thamm T, Gong E, Ouyang J, Christensen S,
Marks MP, Lansberg MG, Albers GW, Zaharchuk G. Tissue at risk and isch-
emic core estimation using deep learning in acute stroke. Am J Neuroradiol.
2021;42:1030–1037. doi: 10.3174/ajnr.A7081
56. Kasasbeh AS, Christensen S, Parsons MW, Campbell B, Albers GW, Lansberg MG.
Artificial neural network computer tomography perfusion prediction of ischemic
core. Stroke. 2019;50:1578–1581. doi: 10.1161/STROKEAHA.118.022649
57. Chung CC, Chan L, Bamodu OA, Hong CT, Chiu HW. Artificial neural net-
work based prediction of postthrombolysis intracerebral hemorrhage and
death. Sci Rep. 2020;10:20501. doi: 10.1038/s41598-020-77546-5
58. Wang F, Huang Y, Xia Y, Zhang W, Fang K, Zhou X, Yu X, Cheng X, Li G, Wang X,
et al. Personalized risk prediction of symptomatic intracerebral hemorrhage
after stroke thrombolysis using a machine-learning model. Ther Adv Neurol
Disord. 2020;13:175628642090235. doi: 10.1177/1756286420902358
59. Yu Y, Guo D, Lou M, Liebeskind D, Scalzo F. Prediction of hemorrhagic trans-
formation severity in acute stroke from source perfusion MRI. IEEE Trans
Biomed Eng. 2018;65:2058–2065. doi: 10.1109/TBME.2017.2783241
60. Dharmasaroja P, Dharmasaroja PA. Prediction of intracerebral hemor-
rhage following thrombolytic therapy for acute ischemic stroke using
multiple artificial neural networks. Neurol Res. 2012;34:120–128. doi:
10.1179/1743132811Y.0000000067
61. Bentley P, Ganesalingam J, Carlton Jones AL, Mahady K, Epton S, Rinne P,
Sharma P, Halse O, Mehta A, Rueckert D. Prediction of stroke thrombolysis
outcome using CT brain machine learning. NeuroImage Clin. 2014;4:635–
640. doi: 10.1016/j.nicl.2014.02.003
62. Smith WS, Keenan KJ, Lovoi PA. A unique signature of cardiac-induced cra-
nial forces during acute large vessel stroke and development of a predictive
model. Neurocrit Care. 2020;33:58–63. doi: 10.1007/s12028-019-00845-x
63. Stib MT, Vasquez J, Dong MP, Kim YH, Subzwari SS, Triedman HJ, Wang A,
Wang HLC, Yao AD, Jayaraman M, et al. Detecting large vessel occlusion at
multiphase CT angiography by using a deep convolutional neural network.
Radiology. 2020;297:640–649. doi: 10.1148/radiol.2020200334
64. Meijs M, Meijer FJA, Prokop M, Ginneken B van,
Manniesing R. Image-level detection of arterial occlusions in 4D-CTA of acute
stroke patients using deep learning. Med Image Anal. 2020;66:101810. doi:
10.1016/j.media.2020.101810
65. You J, Tsang ACO, Yu PLH, Tsui ELH, Woo PPS, Lui CSM, Leung GKK. Auto-
mated hierarchy evaluation system of large vessel occlusion in acute ischemia
stroke. Front Neuroinformatics. 2020;14:13. doi: 10.3389/fninf.2020.00013
66. Ho KC, Speier W, El-Saden S, Arnold CW. Classifying acute ischemic
stroke onset time using deep imaging features. AMIA Annu Symp Proc.
2018;2017:892–901.
67. Ho KC, Speier W, Zhang H, Scalzo F, El-Saden S, Arnold CW. A machine learning
approach for classifying ischemic stroke onset time from imaging. IEEE Trans
Med Imaging. 2019;38:1666–1676. doi: 10.1109/TMI.2019.2901445
68. Lee H, Lee EJ, Ham S, Lee HB, Lee JS, Kwon SU, Kim JS, Kim N, Kang DW.
Machine learning approach to identify stroke within 4.5 hours. Stroke.
2020;51:860–866. doi: 10.1161/STROKEAHA.119.027611
June 2023   1515
 Akay et al
69. Zhang H, Polson JS, Nael K, Salamon N, Yoo B, El-Saden S, Scalzo F, Speier W,
CLINICAL AND POPULATION
Arnold CW. Intra-domain task-adaptive transfer learning to determine acute
ischemic stroke onset time. Comput Med Imaging Graph. 2021;90:101926.
doi: 10.1016/j.compmedimag.2021.101926
SCIENCES
70. Ho KC, Speier W, El-Saden S, Liebeskind DS, Saver JL, Bui AAT, Arnold CW.
Predicting discharge mortality after acute ischemic stroke using balanced
data. AMIA Annu Symp Proc. 2014;2014:1787–1796.
71. Li X, Wu M, Sun C, Zhao Z, Wang F, Zheng X, Ge W, Zhou J, Zou J. Using machine
learning to predict stroke-associated pneumonia in Chinese acute ischaemic
stroke patients. Eur J Neurol. 2020;27:1656–1663. doi: 10.1111/ene.14295
72. Li X, Pan X, Jiang C, Wu M, Liu Y, Wang F, Zheng X, Yang J, Sun C,
Zhu Y, et al. Predicting 6-month unfavorable outcome of acute ischemic
stroke using machine learning. Front Neurol. 2020;11:539509. doi:
10.3389/fneur.2020.539509
73. Qiu W, Kuang H, Nair J, Assis Z, Najm M, McDougall C, McDougall B,
Chung K, Wilson AT, Goyal M, et al. Radiomics-based intracranial thrombus
features on CT and CTA predict recanalization with intravenous alteplase in
patients with acute ischemic stroke. Am J Neuroradiol. 2019;40:39–44. doi:
10.3174/ajnr.A5918
74. Wang K, Shou Q, Ma SJ, Liebeskind D, Qiao XJ, Saver J,
Salamon N, Kim H, Yu Y, Xie Y, et al. Deep learning detection of penum-
bral tissue on arterial spin labeling in stroke. Stroke. 2020;51:489–497. doi:
10.1161/STROKEAHA.119.027457
75. Hofmeister J, Bernava G, Rosi A, Vargas MI, Carrera E, Montet X, Burgermeister S,
Poletti PA, Platon A, Lovblad KO, et al. Clot-based radiomics predict a mechanical
thrombectomy strategy for successful recanalization in acute ischemic stroke.
Stroke. 2020;51:2488–2494. doi: 10.1161/STROKEAHA.120.030334
Downloaded from http://ahajournals.org by on August 26, 2023
1516   June 2023
AI for Clinical Decision Support in Acute Stroke
76. Fu B, Qi S, Tao L, Xu H, Kang Y, Yao Y, Yang B, Duan Y, Chen H. Image
patch-based net water uptake and radiomics models predict malignant
cerebral edema after ischemic stroke. Front Neurol. 2020;11:609747. doi:
10.3389/fneur.2020.609747
77. Sung SM, Kang YJ, Cho HJ, Kim NR, Lee SM, Choi BK, Cho G. Predic-
tion of early neurological deterioration in acute minor ischemic stroke by
machine learning algorithms. Clin Neurol Neurosurg. 2020;195:105892. doi:
10.1016/j.clineuro.2020.105892
78. Velagapudi L, Mouchtouris N, Schmidt RF, Vuong D, Khanna O,
Sweid A, Sadler B, Saiegh FA, Gooch MR, Jabbour P, et al. A machine learn-
ing approach to first pass reperfusion in mechanical thrombectomy: predic-
tion and feature analysis. J Stroke Cerebrovasc Dis. 2021;30:105796.
79. Kapoor S, Narayanan A. Leakage and the reproducibility crisis in ML-based
science. 2022. http://arxiv.org/abs/2207.07048
80. Ding L, Liu C, Li Z, Wang Y. Incorporating artificial intelligence
into stroke care and research. Stroke. 2020;51:e351–e354. doi:
10.1161/STROKEAHA.120.031295
81. Hazan E, Klivans A, Yuan Y. Hyperparameter optimization: a spectral
approach. ArXiv170600764 Cs Math Stat. 2018 [cited 2022 Mar 2]. http://
arxiv.org/abs/1706.00764
82. Directorate-General for Communications Networks C and T (European
C, Grupa ekspertów wysokiego szczebla ds. sztucznej inteligencji. Ethics
guidelines for trustworthy AI. LU: Publications Office of the European Union;
2019 [cited 2022 Mar 2]. https://data.europa.eu/doi/10.2759/346720
83. Higgins D, Madai VI. From bit to bedside: a practical framework for arti-
ficial intelligence product development in healthcare. Adv Intell Syst.
2020;2:2000052.
Stroke. 2023;54:1505–1516. DOI: 10.1161/STROKEAHA.122.041442