Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?

casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023


Clinical Neurology: Research Article

Eur Neurol 2022;85:273–279 Received: October 26, 2021


Accepted: January 24, 2022
DOI: 10.1159/000522254 Published online: March 29, 2022

Practical Machine Learning Model to


Predict the Recovery of Motor Function
in Patients with Stroke
Jeoung Kun Kim a Zhihan Lv b Donghwi Park c Min Cheol Chang d
aDepartment of Business Administration, School of Business, Yeungnam University, Gyeongsan-Si, Republic of

Korea; bDepartment of Game Design, Faculty of Arts, Uppsala University, Uppsala, Sweden; cDepartment of Physical
Medicine and Rehabilitation, Ulsan University Hospital, University of Ulsan College of Medicine, Ulsan, Republic
of Korea; dDepartment of Rehabilitation Medicine, College of Medicine, Yeungnam University, Daegu, Republic of
Korea

Keywords FAC ≥4 were considered to have a “good” outcome. Results:


Artificial intelligence · Machine learning · Deep neural The area under the curve (AUC) for the DNN model for pre-
network · Stroke · Motor recovery · Prediction dicting motor function was 0.836 for the upper and lower
limb motor functions. For the random forest and logistic re-
gression models, the AUCs were 0.736 and 0.790 for the up-
Abstract per and lower limb motor functions, respectively. The AUCs
Background: Machine learning (ML) is an artificial intelli- for the random forest and logistic regression models were
gence technique in which a system learns patterns and rules 0.741 and 0.795 for the upper and lower limb motor func-
from a given data. Objectives: The objective of the study was tions, respectively. Conclusion: Although we used simple
to investigate the potential of ML for predicting motor re- and common data that can be obtained in clinical practice
covery in stroke patients. Methods: We analyzed data from as variables, our DNN algorithm was useful for predicting
833 consecutive stroke patients using 3 ML algorithms: deep motor recovery of the upper and lower extremities in stroke
neural network (DNN), random forest, and logistic regres- patients during the recovery phase. © 2022 S. Karger AG, Basel
sion. We created a practical ML model using the most com-
mon data measured in almost all rehabilitation hospitals as
input data. Demographic and clinical data, including modi-
fied Brunnstrom classification (MBC) and functional ambula- Introduction
tion classification (FAC), were collected when patients were
transferred to the rehabilitation unit (8–30 days) and 6 Stroke is the second leading cause of death in the el-
months after stroke onset and were used as input data. Mo- derly in high-income countries, and it is one of the lead-
tor outcomes at 6 months after stroke onset of the affected ing causes of serious long-term disability in the adult pop-
upper and lower extremities were classified according to ulation [1]. Many post-stroke patients continue to have a
MBC and FAC, respectively. Patients with an MBC of <5 and
an FAC of <4 at 6 months after stroke onset were considered Jeoung Kun Kim and Zhihan Lv equally contributed to this work as
to have a “poor” outcome, whereas those with MBC ≥5 and first authors.

Karger@karger.com © 2022 S. Karger AG, Basel Correspondence to:


www.karger.com/ene Donghwi Park, bdome @ hanmail.net
Min Cheol Chang, wheel633 @ gmail.com
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
Data of 1,328 stroke patients

Data of 486 stroke patients were


excluded after applying inclusion
and exclusion criteria
A total of data of 842 patients
were analyzed

Training set Validation Test set


set

429 patients (MBC) 129 patients (MBC) 56 patients (MBC)


583 patients (FAC) 175 patients (FAC) 75 patients (FAC)

Feature extraction Feature extraction Feature extraction

Trained model Best model Model performance


Training
performance validation re-validation

Hyper-parameter tuning and training refinement Final ML model

Fig. 1. The overall process of analysis of data with ML. MBC, modified Brunnstrom classification; FAC, func-
tional ambulation score; ML, machine learning.

residual disability even after acute stroke management [2, field has produced promising results. In particular, the
3]. Appropriate rehabilitative treatment can improve complex and unpredictable nature of human physiology
functional recovery and reduce post-stroke disability [4, has proven to be better explained by ML algorithms in
5]. Knowledge of motor function prognosis is useful for many circumstances [13]. Previous studies in the field of
clinicians in determining the most appropriate and effec- stroke rehabilitation demonstrated the potential of ML to
tive rehabilitation strategy [4]. be used in motor function prediction [13–15]. However,
To date, several prognostic scoring systems, such as because the input data used in previous ML studies were
Acute ASTRAL, DRAGON, and SEDAN scores, have diverse and the data forms used by hospitals in actual
been developed for this purpose [6, 7]. In addition, trans- clinical practice differed, it is difficult to uniformly use it
cranial magnetic stimulation and diffusion tensor trac- in many hospitals. In this study, we created a practical ML
tography studies have been used for predicting motor model using the most common data measured in almost
outcome after stroke [8, 9]. However, in clinical practice, all rehabilitation hospitals as input data.
the aforementioned methods frequently fail to predict the
actual results [10]. Furthermore, because each hospital
Materials and Methods
has a variety of tests and uses different evaluation tools, it
is often difficult to apply the abovementioned predicting This study was approved by the Institutional Review Board of
systems or tools. Therefore, efforts to improve the predic- Yeungnam University Hospital, and informed consent was waived
tion accuracy and the use of commonly used are required. because of the retrospective nature of the study and because the
Machine learning (ML) is an artificial intelligence analysis involved anonymous clinical data. This study included
stroke patients who were admitted to the physical medicine and
technique in which a system learns patterns and rules
rehabilitation department of a single university hospital for stroke
from a given data [11–13]. It has the advantage of detect- rehabilitation between January 2009 and March 2021. The steps
ing potential interactions between many attributes/vari- of the modeling process applied in this study are shown in Figure
ables. The application of this technology in the medical 1.

274 Eur Neurol 2022;85:273–279 Kim/Lv/Park/Chang


DOI: 10.1159/000522254
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
Table 1. Performances comparison for random forest, logistic regression, DNN models

ML model MBC prediction model FAC prediction model

Sample size (patients) 429 for training, 129 for validation, 56 for test, total 614 583 for training, 175 for validation, 75 for test, total 833

Sample zero ratio Train 50.12%, validation 50.45%, test 50.00% Train 45.97%, validation 46.0%, test 46.0%

DNN 3 layers with 256-256-256 neurons, learning rate 4e−06 3 layers with 256-512-1024 neurons, learning rate 5e−06
SGD optimizer, ELU activation, batch size 16, drop out rate 0.2 SGD optimizer, ReLU activation, batch size 32, drop out rate 0.6
Training accuracy: 78.09% Training accuracy: 90.22%
Validation accuracy: 83.72% Validation accuracy: 83.43%
Test accuracy: 80.36% Test accuracy: 78.67%
Validation AUC 0.836 with CI (0.774–0.898) Validation AUC 0.836 with CI (0.782–0.891)
Test AUC 0.804 with CI (0.703–0.904) Test AUC 0.782 with CI (0.687–0.877)

Logistic regression Training accuracy: 75.99% Training accuracy: 71.87%


Validation accuracy: 79.07% Validation accuracy: 78.86%
Test accuracy: 76.79% Test accuracy: 69.33%
Validation AUC 0.790 with CI (0.722–0.858) Validation AUC 0.795 with CI (0.736–0.853)
Test AUC 0.768 with CI (0.664–0.872) Test AUC 0.692 with CI (0.585–0.798)

Random forest 500 estimators 500 estimators


Out-of-bag score estimate: 72.73% Out-of-bag score estimate: 67.75%
Mean validation accuracy score: 73.64% Mean validation accuracy score: 74.29%
Mean test accuracy score: 75.00% Mean test accuracy score: 69.33%
Validation AUC 0.736 with CI (0.660–0.813) Validation AUC 0.741 with CI (0.675–0.806)
Test AUC 0.750 with CI (0.636–0.864) Test AUC 0.684 with CI (0.579–0.790)

ML, machine learning; MBC, modified Brunnstrom classification; FAC, functional ambulation category; DNN, deep neural network; SGD, stochastic gradient
descent; AUC, area under the curve; CI, confidence interval.

Data Collection of the inputs [20]. The random forest algorithm comprises several
The inclusion criteria were as follows: (1) first-ever stroke, (2) decision trees that consist of multiple true or false conditions using
age over 20 years, (3) hemiplegia or hemiparesis following stroke, input variables [21]. The final classification is based on the sum of
(4) clinical data collected within 7–30 days (early stage, day of the decisions made by the decision trees [22]. Logistic regression
transfer, or day of admission to the rehabilitation department) af- is a statistical technique for estimating the causal relationship be-
ter onset, and (5) absence of serious medical complications, such tween a dependent variable with only two values and the indepen-
as pneumonia or cardiac problems (acute coronary syndromes [4 dent variables using a logistic function [23].
patients] and cardiomyopathy [2 patients]) from onset to final Two DNN models were trained for FAC and MBC predictions
evaluation. The exclusion criteria were as follows: (1) other preex- with all variables as inputs to classify the patients’ motor outcomes.
isting brain or spinal cord lesions and (2) presence of other periph- For the DNN, three layers with 256-256-256 neurons (MBC predic-
eral neuropathies that could affect ankle dorsiflexion strength, tion) and 256-512-1024 neurons (FAC prediction) were used. For
such as peripheral polyneuropathy. the random forest model, 500 decision trees were used (Table 1).
The following demographic and clinical data were collected To understand the effects of two demographic variables – sex
when patients were transferred to the rehabilitation unit (8–30 and age, we evaluated performance change for each model which
days after stroke onset): age, sex, type of stroke (ischemic/hemor- does not use sex or age as input data. The evaluation result shows
rhagic), modified Brunnstrom classification (MBC), functional that the DNN model with all data outperformed the other two
ambulation score (FAC), and Medical Research Council (MRC) models (without sex or age) on both validation and test datasets.
score for muscle strength of shoulder abductor, elbow flexor, fin- Logistic regression and random forest model show only minor dif-
ger flexor, finger extensor, hip flexor, knee extensor, and ankle ferences or no difference without sex or age. The details are shown
dorsiflexor of the affected side [16, 17]. We selected these input in Tables 2 and 3.
variables because they are most commonly and easily collected We categorized the output variables as “good” and “poor” in
when stroke patients are admitted or visit the hospital for reha- the upper and lower extremities. In the lower extremity, patients
bilitation. with an FAC of <4 at 6 months after stroke onset were considered
We used three ML algorithms: deep neural network (DNN), to have a “poor” outcome, while those with scores ≥4 were consid-
random forest, and logistic regression [18]. The DNN algorithm ered to have a “good” outcome. In the upper extremity, patients
consists of layers of interconnected artificial neurons [19]. An ar- with an MBC score of <5 at 6 months after stroke onset were con-
tificial neuron is designed based on the biological neuron and re- sidered to have a “poor” outcome, while those with scores ≥5 were
ceives multiple inputs multiplied by weights and outputs the sum considered to have a “good” outcome.

Artificial Intelligence for Stroke Eur Neurol 2022;85:273–279 275


Rehabilitation DOI: 10.1159/000522254
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
Table 2. Modified Brunnstrom classification prediction model performance change comparison

ML model Base model with all variables Model without sex Diff Model without age Diff

DNN Training accuracy: 78.09% Training accuracy: 80.89% Training accuracy: 85.08%
Validation accuracy: 83.72% Validation accuracy: 83.72% 0% Validation accuracy: 80.62% −3.1%
Test accuracy: 80.36% Test accuracy: 78.57% −1.79% Test accuracy: 78.57% −1.79%

Logistic Training accuracy: 75.99% Training accuracy: 76.22% Training accuracy: 75.52%
regression Validation accuracy: 79.07% Validation accuracy: 77.52% −1.55% Validation accuracy: 78.79% −0.28%
Test accuracy: 76.79% Test accuracy: 75.00% −1.79% Test accuracy: 78.57% +1.78%

Random Out-of-bag score estimate: 72.73% Out-of-bag score estimate: 70.16% Out-of-bag score estimate: 72.49%
forest Mean validation accuracy score: 73.64% Mean validation accuracy score: 73.64% 0% Mean validation accuracy score: 75.19% +1.55%
Mean test accuracy score: 75.00% Mean test accuracy score: 75.00% 0% Mean test accuracy score: 76.79% +1.79%

ML, machine learning; DNN, deep neural network.

Table 3. Functional ambulation category prediction model performance change comparison

ML model Base model with all variables Model without sex Diff Model without age Diff

DNN Training accuracy: 90.22% Training accuracy: 79.93%% Training accuracy: 79.07%
Validation accuracy: 83.43% Validation accuracy: 81.17% −2.26% Validation accuracy: 80.00% −3.43%
Test accuracy: 78.67% Test accuracy: 69.33% −9.33% Test accuracy: 72.00% −6.67%

Logistic Training accuracy: 71.87% Training accuracy: 71.87% Training accuracy: 69.98%
regression Validation accuracy: 78.86% Validation accuracy: 78.86% 0% Validation accuracy: 77.71% −1.15%
Test accuracy: 69.33% Test accuracy: 69.33% 0% Test accuracy: 70.67% +1.34%

Random Out-of-bag score estimate: 67.75% Out-of-bag score estimate: 66.90% Out-of-bag score estimate: 67.41%
forest Mean validation accuracy score: 74.29% Mean validation accuracy score: 73.14% −1.15% Mean validation accuracy score: 77.14% +2.85%
Mean test accuracy score: 69.33% Mean test accuracy score: 66.67% −2.66% Mean test accuracy score: 68.00% +1.33%

ML, machine learning; DNN, deep neural network.

For this study, data from 842 patients (mean age: 60.8 ± 13.2 culated. The confidence interval (CI) for AUC was calculated using
years; 480 males, 353 females; stroke onset: 16.3 ± 6.6 days) were the approach used by DeLong et al. [24].
analyzed. To develop a model for predicting motor outcome of upper
extremity, 614 patients’ data (mean age 61.0 ± 13.0 years; 349 males,
265 females; stroke onset: 16.4 ± 6.5 days) were used. To prevent
Results
overfitting of the models, 70% (n = 429, poor: good = 50.1%: 49.9%),
21% (n = 129, poor: good = 50.5%: 49.7%), and 9% (n = 56, poor: good
= 50.0%: 50.0%) of these data were included in the training, valida- In the prediction of the motor outcome of the upper
tion, and test sets, respectively. To develop a model for predicting extremity, the AUC of the validation dataset for the DNN
motor outcome of lower extremity, 833 patients’ data (mean age: 60.8 model was 0.836 (95% CI, 0.774–0.898). For the random
± 13.2 years; 480 males, 353 females; stroke onset: 16.3 ± 6.6 days)
forest and logistic regression models, the AUC of the val-
were used: 70% (n = 583, poor : good = 46.0%: 54.0%) for training set,
21% (n = 175, poor: good = 46.3%: 53.7%) for validation set, and 9% idation dataset was 0.736 (95% CI, 0.660–0.813) and 0.790
(n = 75, poor: good = 45.3%: 54.7%) for test sets. TensorFlow version (95% CI, 0.722–0.858), respectively (Table 1; Fig. 2).
2.4.1 (Google, Mountain View, CA, USA) and the scikit-learn toolkit As for the motor outcome of the lower extremity, the
version 0.24.1 were used to train the ML models. AUC of the validation dataset for the DNN model was
0.836 (95% CI, 0.782–0.891). For the random forest and
Statistical Analysis
Statistical analyses were performed using Python 3.8.8 and logistic regression models, the AUC was 0.741 (95% CI,
scikit-learn version 0.24.1. Receiver operating characteristic curve 0.675–0.806) and 0.795 (95% CI, 0.736–0.853), respec-
analysis was employed, and area under the curve (AUC) was cal- tively (Table 1; Fig. 2).

276 Eur Neurol 2022;85:273–279 Kim/Lv/Park/Chang


DOI: 10.1159/000522254
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
Color version available online
MBC validation receiver operating characteristic curve FAC validation receiver operating characteristic curve
1.0 1.0

0.8 0.8
True positive rate

True positive rate


0.6 0.6

0.4 0.4

0.2 Deep neural network, AUC = 0.836 0.2 Deep neural network, AUC = 0.836
Logistic regression, AUC = 0.790 Logistic regression, AUC = 0.795
Random forest, AUC = 0.736 Random forest, AUC = 0.741
0 0
0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1.0
False positive rate False positive rate

Fig. 2. Receiver operating characteristic curves for validation data. AUC, area under the curve.

Discussion of daily living in acute stroke patients. The researchers


used data from 313 patients to predict the Bathel index
In predicting both motor outcomes of the upper and score at discharge using ML methods, such as support
lower extremities in stroke patients, the DNN model vector machine, and they obtained an AUC of 0.777. Lin
showed the highest AUC, followed by the logistic regres- et al. [14] used several data, such as mRS, BI, functional
sion model and random forest. The AUC for the DNN oral intake scale, mini nutrition assessment, European
model for predicting motor function was 0.836 for the quality of life five dimensions questionnaire, instrumen-
upper and lower limb motor functions. For the random tal activities of daily living scale, Berg balance test, gait
forest and logistic regression models, the AUCs were un- speed, 6-min walk test, Fugl-Meyer upper extremity as-
der 0.8. sessment, modified Fugl-Meyer sensory assessment,
The most noticeable improvements usually occur mini-mental state examination, motor activity log amount
within the first few weeks of stroke onset, after which the of use, and concise Chinese aphasia test as input data [14].
rate of improvement slows and reaches a relatively stable Wang et al. [15] developed a prognostic model of func-
state after 3 months [25, 26]. Within 3 months after stroke tional outcome using data from 333 patients with prima-
onset, 70% recovery in motor function is known to occur. ry ICH. They utilized Auto-WEKA 2.0, which uses a se-
After 6 months, recovery usually reaches its limit and en- quential model-based algorithm configuration to deter-
ters a chronic phase [25, 26]. Therefore, in this study, we mine the class with the best performance on the given
used the FAC and MBC score at 6 months after stroke data. Functional scores at 1 and 6 months after onset,
onset as indicators of recovery of the upper and lower ex- evaluated with the mRS, were used as the outcome data,
tremities. and an AUC of 0.899 at 1 month and 0.917 at 6 months
In previous studies, ML models were used to predict was obtained. To predict the outcome, they used demo-
functional improvement in stroke patients. For example, graphic attributes (age, medical history of hypertension
Heo et al. [13] predicted the modified Rankin scale (mRS) and diabetes, blood pressure, and level of consciousness),
score using the DNN with 2,604 acute ischemic stroke radiologic attributes (volume and location of hematoma,
subjects and reported an AUC of 0.888. The researchers presence of intraventricular hemorrhage, ventricle com-
used six variables to calculate the ASTRAL score as input pression, and midline structure shift), and laboratory at-
data to predict the outcome. Lin et al. [14] investigated tributes (serum glucose level, aspartate aminotransferase,
whether ML models can predict the recovery of activities alanine aminotransferase, blood urea nitrogen, creati-

Artificial Intelligence for Stroke Eur Neurol 2022;85:273–279 277


Rehabilitation DOI: 10.1159/000522254
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
nine, the blood urea nitrogen/creatinine ratio, glycosyl- regression. Multiple layers of complex networks in DNN
ated hemoglobin, the complete blood count, triglyceride, would be efficient for representing the complex charac-
total cholesterol, C-reactive protein, uric acid, prothrom- teristics of the clinical outcomes of a stroke patient. Our
bin time, activated partial thrombin time, and hypersen- study is limited in that it is a single-center study and
sitive C-reactive protein) as input data. should be validated with data from more sources from
Unlike the abovementioned previous studies, we at- other hospitals. Also, we did not use data on comorbidi-
tempted to develop a practical ML model using the most ties as input data, their addition is expected to increase the
commonly and easily measured clinical data. The MRCs prediction accuracy.
in each muscle group were measured in almost all reha-
bilitation hospitals. MBC and FAC are easily checked by
physiatrists and physical therapists [27]. Furthermore, in Statement of Ethics
a previous Delphi study that evaluated what medical in-
This study protocol was reviewed and approved by the Institu-
formation is essential for physicians to improve the effi-
tional Review Board of Yeungnam University Hospital, approval
ciency of transferring medical records for stroke patients number (2019-10-008). Informed consent was waived by the Insti-
undergoing interhospital transfer, variables such as age, tutional Review Board of Yeungnam University Hospital because
sex, type of stroke (ischemic/hemorrhagic), MRC scores, of the retrospective nature of the study and because the analysis
MBC, and FAC, which were used in our study, were re- involved anonymous clinical data.
ported to be indispensable information [28]. Therefore,
we used these variables as input data to develop a practical
ML algorithm for predicting post-stroke motor out- Conflict of Interest Statement
comes. Previously developed ML algorithms for predict- The authors have no conflicts of interest to declare.
ing post-stroke functional or motor outcomes used vari-
ous measurements as input data, which could increase the
prediction accuracy of the ML algorithm. However, neu- Funding Sources
rology and rehabilitation departments of hospitals evalu-
ate stroke patients’ states using different tools and collect The present study was supported by a National Research Foun-
different clinical data. Accordingly, it is practically inap- dation of Korea grant funded by the Korean government (grant
propriate to use several various data as input data for pre- No. NRF-2019M3E5D1A02068106).
viously developed algorithm. In addition, there are time
constraints for clinicians to collect all input data used in
previous studies, making it less effective. Since our ML Author Contributions
algorithm used only essential data that can be easily J.K.K., Z.L., D.P., and M.C.C. collected the data, designed the
checked as inputs, its use by clinicians is convenient and study, performed the statistical analysis, and drafted the manu-
practical. script.
Although we used simple and common data that can
be obtained much more easily than previous studies as
variables [13–15], the current study demonstrated that Data Availability Statement
ML models could be useful for predicting the recovery of
The corresponding author will provide the data on reasonable
upper and lower extremities in acute stroke patients.
request.
Considering that AUCs of 0.7–0.8, 0.8–0.9, and >0.9 are
generally considered acceptable, excellent, and outstand-
ing, the ability of the ML models used in this study to pre-
References 1 Donkor ES. Stroke in the 21(st) century: a
dict the recovery of the upper extremity and lower ex- snapshot of the burden, epidemiology, and
tremity were excellent, with the DNN model outperform- quality of life. Stroke Res Treat. 2018; 2018:
ing the other models (random forest and logistic 3238165.
2 Hillis AE, Tippett DC. Stroke recovery: sur-
regression models). prising influences and residual consequences.
In conclusion, our results showed that DNN using Adv Med. 2014;2014:378263.
simple and common data as input variables is an excellent 3 Singh RJ, Chen S, Ganesh A, Hill MD. Long-
term neurological, vascular, and mortality
predictor of motor outcomes after stroke. The DNN was outcomes after stroke. Int J Stroke. 2018;
found to be better than decision tree forest and logistic 13(8):787–96.

278 Eur Neurol 2022;85:273–279 Kim/Lv/Park/Chang


DOI: 10.1159/000522254
Downloaded from http://karger.com/ene/article-pdf/85/4/273/3717208/000522254.pdf?casa_token=sPLedTK4O2kAAAAA:ag77ESPWKnZPGeXOQbHWZ2MD0cwp5ibNosdFxkgRF2uSMk6p45g5tT2-d-Tqi7RoeXgjfb-9zgQ by ZZduplicate-BCU Lausanne user on 27 June 2023
4 Hatem SM, Saussez G, Della Faille M, Prist V, 12 Lee JT, Park E, Hwang JM, Jung TD, Park D. 21 Nasejje JB, Mwambi H. Application of ran-
Zhang X, Dispa D, et al. Rehabilitation of mo- Machine learning analysis to automatically dom survival forests in understanding the de-
tor function after stroke: a multiple system- measure response time of pharyngeal swal- terminants of under-five child mortality in
atic review focused on techniques to stimulate lowing reflex in videofluoroscopic swallowing Uganda in the presence of covariates that sat-
upper extremity recovery. Front Hum Neuro- study. Sci Rep. 2020;10(1):14735. isfy the proportional and non-proportional
sci. 2016;10:442. 13 Heo J, Yoon JG, Park H, Kim YD, Nam HS, hazards assumption. BMC Res Notes. 2017;
5 Chien SH, Sung PY, Liao WL, Tsai SW. A Heo JH. Machine learning-based model for 10(1):459.
functional recovery profile for patients with prediction of outcomes in acute stroke. 22 Shah SA, Brown P, Gimeno H, Lin JP, Mc-
stroke following post-acute rehabilitation Stroke. 2019;50(5):1263–5. Clelland VM. Application of machine learn-
care in Taiwan. J Formos Med Assoc. 2020; 14 Lin WY, Chen CH, Tseng YJ, Tsai YT, Chang ing using decision trees for prognosis of deep
119(1 Pt 2):254–9. CY, Wang HY, et al. Predicting post-stroke brain stimulation of globus pallidus internus
6 Ntaios G, Gioulekas F, Papavasileiou V, Strbi- activities of daily living through a machine for children with dystonia. Front Neurol.
an D, Michel P. ASTRAL, DRAGON and SE- learning-based approach on initiating reha- 2020;11:825.
DAN scores predict stroke outcome more ac- bilitation. Int J Med Inform. 2018; 111: 159– 23 Park HA. An introduction to logistic regres-
curately than physicians. Eur J Neurol. 2016; 64. sion: from basic concepts to interpretation
23(11):1651–7. 15 Wang HL, Hsu WY, Lee MH, Weng HH, with particular attention to nursing domain. J
7 Ntaios G, Faouzi M, Ferrari J, Lang W, Vem- Chang SW, Yang JT, et al. Automatic ma- Korean Acad Nurs. 2013;43(2):154–64.
mos K, Michel P. An integer-based score to chine-learning-based outcome prediction in 24 DeLong ER, DeLomg DM, Clarke-Pearson
predict functional outcome in acute ischemic patients with primary intracerebral hemor- DL. Comparing the area under two or more
stroke: the ASTRAL score. Neurology. 2012; rhage. Front Neurol. 2019;10:910. correlated receiver operating characteristic
78(24):1916–22. 16 Jang SH, Chang MC. Motor outcomes of pa- curves: a nonparametric approach. Biomet-
8 Jang SH, Ahn YH, Kim SH, Chang CH. Cor- tients with a complete middle cerebral artery rics. 2998;44:837–45.
ticospinal tract restoration: combined study territory infarct. Neural Regen Res. 2013; 25 Lee KB, Lim SH, Kim KH, Kim KJ, Kim YR,
of diffusion tensor tractography, functional 8(20):1892–7. Chang WN, et al. Six-month functional re-
MRI, and transcranial magnetic stimulation. 17 Chang MC, Park SW, Lee BJ, Park D. Rela- covery of stroke patients: a multi-time-point
J Comput Assist Tomogr. 2007;31(6):901–4. tionship between recovery of motor function study. Int J Rehabil Res. 2015;38(2):173–80.
9 Kim BR, Moon WJ, Kim H, Jung E, Lee J. and neuropsychological functioning in cere- 26 Smith MC, Byblow WD, Barber PA, Stinear
Transcranial magnetic stimulation and diffu- bral infarction patients: the importance of so- CM. Proportional recovery from lower limb
sion tensor tractography for evaluating am- cial functioning in motor recovery. J Integr motor impairment after stroke. Stroke. 2017;
bulation after stroke. J Stroke. 2016; 18(2): Neurosci. 2020;19(3):405–11. 48(5):1400–3.
220–6. 18 Kong Y, Yu T. A deep neural network model 27 Chang MC, Lee BJ, Joo NY, Park D. The pa-
10 Jang SH, Choi BY, Chang CH, Kim SH, Chang using random forest to extract feature repre- rameters of gait analysis related to ambulatory
MC. Prediction of motor outcome based on sentation for gene expression data classifica- and balance functions in hemiplegic stroke
diffusion tensor tractography findings in tha- tion. Sci Rep. 2018;8(1):16477. patients: a gait analysis study. BMC Neurol.
lamic hemorrhage. Int J Neurosci. 2013; 19 Sutoko S, Masuda A, Kandori A, Sasaguri H, 2021;21(1):38.
123(4):233–9. Saito T, Saido TC, et al. Early identification of 28 Kim JK, Hau YS, Kwak S, Chang MC. Essen-
11 Wang J, Zhang X, Cheng L, Luo Y. An over- Alzheimer’s disease in mouse models: appli- tial medical information for stroke patients
view and metanalysis of machine and deep cation of deep neural network algorithm to undergoing interhospital transfer: a delphi
learning-based CRISPR gRNA design tools. cognitive behavioral parameters. iScience. study. Am J Phys Med Rehab. 2021; 100(4):
RNA Biol. 2020;17(1):13–22. 2021;24(3):102198. 354–8.
20 Renganathan V. Overview of artificial neural
network models in the biomedical domain.
Bratisl Lek Listy. 2019;120(7):536–40.

Artificial Intelligence for Stroke Eur Neurol 2022;85:273–279 279


Rehabilitation DOI: 10.1159/000522254

You might also like