STARTING-SICH Nomogram to Predict

Symptomatic Intracerebral Hemorrhage After

Intravenous Thrombolysis for Stroke
Manuel Cappellari, MD; Gianni Turcato, MD; Stefano Forlivesi, MD; Cecilia Zivelonghi, MD;
Paolo Bovi, MD; Bruno Bonetti, MD, PhD; Danilo Toni, MD, PhD

Background and Purpose—Symptomatic intracerebral hemorrhage (sICH) is a rare but the most feared complication of
intravenous thrombolysis for ischemic stroke. We aimed to develop and validate a nomogram for individualized prediction
of sICH in intravenous thrombolysis–treated stroke patients included in the multicenter SITS-ISTR (Safe Implementation
of Thrombolysis in Stroke-International Stroke Thrombolysis Register).
Methods—All patients registered in the SITS-ISTR by 179 Italian centers between May 2001 and March 2016 were
originally included. The main outcome measure was sICH per the European Cooperative Acute Stroke Study II
definition (any type of intracerebral hemorrhage with increase of ≥4 National Institutes of Health Stroke Scale score
points from baseline or death <7 days). On the basis of multivariate logistic model, the nomogram was generated.
We assessed the discriminative performance by using the area under the receiver-operating characteristic curve and
calibration of risk prediction model by using the Hosmer–Lemeshow test.
Results—A total of 15 949 patients with complete data for generating the nomogram was randomly dichotomized into
training (3/4; n=12 030) and test (1/4; n=3919) sets. After multivariate logistic regression, 10 variables remained
independent predictors of sICH to compose the STARTING-SICH (systolic blood pressure, age, onset-to-treatment
time for thrombolysis, National Institutes of Health Stroke Scale score, glucose, aspirin alone, aspirin plus
clopidogrel, anticoagulant with INR ≤1.7, current infarction sign, hyperdense artery sign) nomogram. The area under
the receiver-operating characteristic curve of STARTING-SICH was 0.739. Calibration was good (P=0.327 for the
Hosmer–Lemeshow test).
Conclusions—The STARTING-SICH is the first nomogram developed and validated in a large SITS-ISTR cohort for
Downloaded from http://ahajournals.org by on September 29, 2023

individualized prediction of sICH in intravenous thrombolysis–treated stroke patients.   (Stroke. 2018;49:397-404.

DOI: 10.1161/STROKEAHA.117.018427.)
Key Words: calibration ◼ clopidogrel ◼ hemorrhage ◼ nomograms ◼ stroke

S ymptomatic intracerebral hemorrhage (sICH) is a rare but

the most feared complication of intravenous thromboly-
sis for ischemic stroke. Several prognostic scores have been
designed in the past few years to predict the risk of sICH after
intravenous thrombolysis. The SITS (Safe Implementation of
Treatments in Stroke)-SICH and SEDAN (blood sugar, early
infarct signs and hyperdense cerebral artery sign, age, and
National Institutes of Health Stroke Scale [NIHSS]) scores
have been validated in the large multicenter SITS-ISTR (Safe
Implementation of Treatments in Stroke-International Stroke
Thrombolysis Register).1,2 However, the performance of these
scores for individualized prediction of sICH is limited because
of the dichotomization/categorization of discrete/continuous
variables such as blood glucose, NIHSS score, age, blood pres-
sure (BP), and onset-to-treatment time for thrombolysis, which
are the strongest predictors of sICH after intravenous throm-
bolysis.3 Several studies have described a better performance
of nomograms compared with scores that use risk grouping.4–8
By using a continuous score, a nomogram is a graphical sta-
tistical instrument that calculates the continuous probability
of a particular outcome for an individual patient. Nomograms
are important components of modern medical decision mak-
ing and have been used in an extensive array of applications,
including cancer, surgery, and other specialties.9–11
The present study was aimed to develop and validate a
nomogram to predict the probability of sICH per the ECASS-II

Continuing medical education (CME) credit is available for this article. Go to http://cme.ahajournals.org to take the quiz.
Received June 16, 2017; final revision received December 2, 2017; accepted December 5, 2017.
From the DAI di Neuroscienze, Azienda Ospedaliera Universitaria Integrata, Verona, Italy (M.C., S.F., C.Z., P.B., B.B.); Emergency Department,
Girolamo Fracastoro Hospital San Bonifacio (Verona), Italy (G.T.); and Dipartimento di Neurologia e Psichiatria, Università degli Studi di Roma “La
Sapienza,” Italy (D.T.).
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.
117.018427/-/DC1.
Correspondence to Manuel Cappellari, MD, USD Stroke Unit, DAI di Neuroscienze, Azienda Ospedaliera Universitaria Integrata, Piazzale A. Stefani 1,
37126 Verona, Italy. E-mail manuel_cappellari@libero.it
© 2018 American Heart Association, Inc.
Stroke is available at http://stroke.ahajournals.org DOI: 10.1161/STROKEAHA.117.018427

397
 398  Stroke  February 2018
(European Cooperative Acute Stroke Study-II) definition
after intravenous thrombolysis for individual stroke patients
included in the large multicenter SITS-ISTR.
Methods
Study Design, Participants, and Procedures
All patients registered in the SITS-ISTR by Italian centers between
May 2001 and March 2016 were originally included. All centers reg-
istering patients are required to accept the rules of participating in the
SITS-ISTR (online-only Data Supplement), including consecutive
registration of all stroke patients receiving intravenous alteplase, irre-
spective of whether treatment is on- or off-label. The data that support
the findings of this study are available from the corresponding author
on reasonable request.
Inclusion Criteria
We included all patients with complete data on all the variables
included in the nomogram and clinical and radiological data to deter-
mine sICH.
Exclusion Criteria
Patients treated with endovascular procedures after intravenous
thrombolysis were excluded from the analysis. In agreement with
the current guidelines,12 age <18 years, onset-to-treatment time for
thrombolysis >270 minutes, systolic BP >185 mm Hg, diastolic BP
>110 mm Hg, blood glucose <50 or >400 mg/dL, prestroke oral anti-
coagulants with international normalized ratio (INR) >1.7, and direct
oral anticoagulants were exclusion criteria for the present study.
Outcome
The main outcome was sICH per the ECASS-II definition: any type
Downloaded from http://ahajournals.org by on September 29, 2023
of intracerebral hemorrhage with an increase of ≥4 NIHSS points
from baseline or leading to death <7 days.13 The secondary outcome
measure was sICH per the SITS-MOST (SITS-Monitoring Study)
definition: a local or remote type 2 parenchymal hemorrhage <22 to
36 hours, or earlier in case of clinical deterioration, in combination
with an increase of ≥4 NIHSS points from baseline, or leading to
death <24 hours.14
Statistical Analysis
The Italian SITS-ISTR cohort was randomly dichotomized into
training and test sets using the statistical software STATA 13.0.1
(StataCorp, College Station, TX). Three quarters of the cohort were
used to develop the prediction model and 1/4 to validate the model.
Differences between the cohorts were explored using the Mann–
Whitney U test for continuous variables. Differences between pro-
portions were assessed by Fisher exact test or the χ2 test, when
appropriate. Continuous variables were reported as median and inter-
quartile range values. Proportions were calculated for categorical
variables, dividing the number of events by the total number exclud-
ing missing/unknown cases.
To generate the nomogram, multivariate logistic regression analy-
sis was performed for predicting the probability of ECASS-II sICH
using a forward stepwise method that included all variables with a
probability value <0.10 in the univariate analysis. Collinearity of
combinations of variables in the training set was evaluated by the
variation inflation factors (<2 being considered nonsignificant) and
condition index (<30 being considered nonsignificant). Regression
coefficients and odds ratios (OR) with 2-sided 95% confidence inter-
vals (CI) for each of the variables included in the model were finally
calculated.
To discriminate patients with and without sICH, the predictive
accuracy of the nomogram model was assessed by calculation of the
area under the receiver-operating characteristic curve (AUC-ROC).
Calibration of the risk prediction model was assessed in the test
cohort by the plot comparing the observed probability of ECASS-II
sICH according to the total score of the nomogram against the pre-
dicted probability based on the nomogram and by using the Hosmer–
Lemeshow test that assesses whether or not the observed event rates
matched the expected rates in subgroups of patients.
The overall predictive and discriminative performance of the
nomogram for predicting the probability of ECASS-II sICH in the
test cohort was compared with SITS-SICH1 and SEDAN15 scores
(online-only Data Supplement) by using the estimate of the AUC-
ROC. The contribution of the nomogram for predicting the risk of
ECASS-II sICH compared with SITS-SICH and SEDAN scores
was assessed in the test cohort by net reclassification improvement
method that evaluates the proportion of individuals moving accu-
rately or inaccurately from one risk category to another.16,17 In the
absence of generally agreed thresholds, we used <3% for low risk,
3% to 6% for average risk, and >6% for high risk according to the
current mean rate of ECASS-II sICH (ie, 4.5%) calculated on data
of all patient files entered in the world SITS-Thrombolysis registry
between December 2002 and November 2016 (http://www.sitsinter-
national.org/registries/sits-thrombolysis). We also used different risk
categories in relation to the mean sICH rate observed in our database.
Results
Of 25 808 patients registered in the SITS-ISTR cohort by 179
Italian centers, a final number of 15 949 patients were included
in the study. Details of the patients excluded from the analysis
are provided in the online-only Data Supplement. The clini-
cal characteristics of the patients included and excluded in the
study are listed in Table I in the online-only Data Supplement.
The clinical characteristics of the patients in the training
(n=12 030) and test (n=3919) cohorts are shown in the Table.
No significant differences were noted between the 2 cohorts
of patients. The proportion of patients with ECASS-II sICH
was 3.7% in the training cohort and 3.5% in the test cohort.
The proportion of patients with SITS-MOST sICH was 1.7%
in both cohorts.
Seventeen variables (age, hypertension, diabetes mellitus,
hyperlipidemia, smoking current, atrial fibrillation, conges-
tive heart failure, prestroke antithrombotic therapy with aspi-
rin alone, aspirin plus clopidogrel, or oral anticoagulant with
INR ≤1.7, prestroke modified Rankin Scale score >0, baseline
NIHSS score, hyperdense artery sign, current infarction sign,
systolic BP, glucose, and onset-to-treatment time for thrombol-
ysis) entered the logistic regression model. Five noncategorical
variables and 5 categorical variables remained independent pre-
dictors of ECASS-II sICH to compose the STARTING-SICH
(systolic BP [OR, 1.009; 95% CI, 1.004–1.014; P=0.001],
age [OR, 1.019; 95% CI, 1.009–1.028; P<0.001], onset-to-
treatment time [OR, 1.002; 95% CI, 1.001–1.005; P=0.019],
NIHSS score [OR, 1.050; 95 CI, 1.034–1.066; P<0.001],
glucose [OR, 1.006; 95% CI, 1.004–1.008; P<0.001], aspi-
rin alone [OR, 1.537; 95% CI, 1.251–1.889; P<0.001] or plus
clopidogrel [OR, 2.232; 95% CI, 1.264–3.941; P=0.006], oral
anticoagulant with INR ≤1.7 [OR, 2.065; 95% CI, 1.302–
3.274; P=0.002], current infarction sign [OR, 1.444; 95% CI,
1.302–3.274; P=0.012], hyperdense artery sign [OR, 1.390;
95% CI, 1.118–1.730; P=0.003]) nomogram (Table II in the
online-only Data Supplement). No significant statistical col-
linearity was observed for any of the 10 variables (Tables III
and IV in the online-only Data Supplement).
The nomogram was created by assigning a graphic pre-
liminary score to each of the 10 predictors with a point range
from 0 to 10, which was then summed to generate the total
 Cappellari et al   Nomogram to Predict sICH After Thrombolysis   399

Table. Demographics and Clinical Characteristics in the Table. Continued

Training and Test Cohorts
Training Cohort Test Cohort
Training Cohort Test Cohort (n=12 030) (n=3919) P Value
(n=12 030) (n=3919) P Value
sICH, n (%)
Demographics
 ECASS-II definition 440 (3.7) 138 (3.5) 0.728
 Age, y, median (IQR) 72 (62–78) 71 (62–78) 0.384
 SITS-MOST definition 207 (1.7) 65 (1.7) 0.827
 Age >80 y, n (%) 1668 (13.9) 531 (13.5) 0.630 Results are expressed as number (%) or median (IQR). BP indicates
 Male sex, n (%) 6742 (56.0) 2226 (56.8) 0.407 blood pressure; ECASS-II, European Cooperative Acute Stroke Study-II; INR,
international normalized ratio; IQR, interquartile range; mRS, modified Rankin
Medical history, n (%) Scale; NIHSS, National Institutes of Health Stroke Scale; OTT, onset to treatment;
 Hypertension 7844 (65.2) 2522 (64.4) 0.355 SEDAN, blood sugar, early infarct signs and hyperdense cerebral artery sign,
age, and NIHSS; sICH, symptomatic intracerebral hemorrhage; and SITS, Safe
 Diabetes mellitus 2015 (16.7) 663 (16.9) 0.807 Implementation of Thrombolysis in Stroke.
 Hyperlipidemia 3382 (28.1) 1119 (28.6) 0.596
 Smoking current 2223 (18.5) 725 (18.5) 0.982
score, finally converted into an individual probability of
sICH after thrombolysis (from 0% to 100%; online-only Data
 Previous stroke 1064 (8.8) 345 (8.8) 0.947
Supplement). The STARTING-SICH nomogram is shown in
 Atrial fibrillation 2225 (18.5) 713 (18.2) 0.686 Figure 1 taking into account the approximation of all the vari-
 Congestive heart failure 789 (6.6) 239 (6.1) 0.330 ables that are graphed without decimal. An example of how
to use the nomogram is provided in the Figure I in the online-
 Aspirin 3776 (31.4) 1233 (31.5) 0.937
only Data Supplement.
 Dipyridamole 56 (0.5) 14 (0.4) 0.404 The AUC-ROC of the STARTING-SICH nomogram for
 Clopidogrel 542 (4.5) 169 (4.3) 0.654 predicting the probability of ECASS-II sICH was 0.699 (95%
 Other antiplatelets 480 (4.0) 170 (4.3) 0.353
CI, 0.676–0.723) in the training cohort. The model was inter-
nally validated using 20 000 bootstrap samples to calculate the
 Oral anticoagulant with
294 (2.4) 100 (2.6) 0.725 discrimination with accuracy of 0.739 (95% CI, 0.698–0.779).
INR ≤1.7
The model was validated in the test cohort with AUC-ROC
Baseline data value of 0.739 (95% CI, 0.699–0.780).
Figure 2 displays a calibration plot, comparing the pre-
Downloaded from http://ahajournals.org by on September 29, 2023

 Prestroke mRS score

1842 (15.7) 598 (15.3) 0.541
>0, n (%) dicted proportion of patients who developed ECASS-II
 NIHSS score, median sICH per nomogram with the proportions observed accord-
11 (7–17) 11 (7–17) 0.171
(IQR) ing to STARTING-SICH total score point in the test set. The
 NIHSS score >25,
Hosmer–Lemeshow goodness-of-fit test comparing predicted
106 (0.9) 34 (0.9) 0.993 and observed rates of sICH showed good calibration of the
n (%)
STARTING-SICH total score (8.058; P=0.327).
 Hyperdense artery
sign, n (%)
2858 (23.8) 954 (24.3) 0.464 Frequencies of ECASS-II sICH in the entire cohort per
point increase of the STARTING-SICH total score taking into
 Current infarction sign,
1093 (9.1) 342 (8.7) 0.520 account the approximation without decimal are outlined in
n (%)
Figure 3. The STARTING-SICH total score ranged from 11 to
 Systolic BP, mm Hg, 31 points resulting into a continuum of probability of observed
150 (133–160) 148 (133–160) 0.654
median (IQR)
sICH from 0% (≤13 points) to 10.9% (27 points) in the 99%
 Diastolic BP, mm Hg,
80 (71–90) 80 (71–90) 0.508
of patients of the entire cohort. Patients with a score of 28
median (IQR) (0.6%), 29 (0.3%), 30 (0.1%), and 31 (0.1%) had a sICH rate
 Glucose, mg/dL, of 13.5%, 17.1%, 21.1%, and 40%, respectively. Frequencies
118 (102–142) 117 (102–141) 0.467
median (IQR) of ECASS-II sICH per point increase of the SITS-SICH and
 Weight, kg, median SEDAN scores are, respectively, outlined in Figures II and III
74 (65–83) 74 (65–82) 0.348 in the online-only Data Supplement.
(IQR)
 Alteplase dose, mg,
In the test set, all patients had available data to calculate the
66 (58–74) 67 (58–74) 0.572 SITS-SICH and SEDAN scores. Compared with the discrimi-
median (IQR)
native performance of the STARTING-SICH nomogram for
 OTT time for
thrombolysis, min, 164 (130–193) 164 (130–190) 0.559
predicting the probability of ECASS-II sICH, the AUC val-
median (IQR) ues of the SEDAN score were lower (0.690; 95% CI, 0.646–
0.735; P=0.020), whereas the AUC values of the SITS-SICH
Scores, median (IQR)
score were similar (0.718; 95% CI, 0.677–0.759; P=0.148;
 SITS-SICH 4 (3–6) 4 (3–5) 0.198 Figure 4).
 SEDAN 1 (1–2) 1 (1–2) 0.501 Compared with SITS-SICH score, the STARTING-SICH
nomogram improved the net reclassification of ECASS-II
(Continued ) sICH risk when it was categorized into <3%, 3% to 6%, and
 400  Stroke  February 2018
Downloaded from http://ahajournals.org by on September 29, 2023

Figure 1. STARTING-SICH (systolic blood pressure, age, onset-to-treatment time for thrombolysis, NIHSS score, glucose, aspirin alone,
aspirin plus clopidogrel, anticoagulant with INR ≤1.7, current infarction sign, hyperdense artery sign) nomogram for predicting the prob-
ability of symptomatic intracerebral hemorrhage (sICH) per the ECASS-II (European Cooperative Acute Stroke Study-II) definition. BP indi-
cates blood pressure; NIHSS, National Institutes of Health Stroke Scale; and OTT, onset to treatment.

>6% (net reclassification improvement: 0.077; P=0.014; Table
V in the online-only Data Supplement). Among subjects who
developed sICH, the STARTING-SICH resulted in the 16.2%
and 17% of individuals being reclassified, respectively, from
low- to average-risk and from average- to high-risk categories,
whereas the 6.3% of individuals was inappropriately reclassi-
fied from high- to average-risk categories. Conversely, among
individuals who did not develop sICH, the STARTING-SICH
resulted in the 13.2% and 2.2% of individuals being reclassi-
fied, respectively, from high- to average-risk and from aver-
age- to low-risk categories, whereas the 2.1%, 0.1%, and 6.5%
of individuals was inappropriately reclassified, respectively,
from low- to average-risk, from low- to high-risk, and from
moderate- to high-risk categories. No significant difference
was observed when we used the risk categories <2%, 2% to
5%, and >5%.
Compared with SEDAN score, the nomogram improved the
net reclassification when the risk was categorized into <2%,
2% to 5%, and >5% (net reclassification improvement: 0.126;
P<0.001; Table VI in the online-only Data Supplement).
Among subjects who developed sICH, the STARTING-SICH
resulted in the 7.2% of individuals being reclassified, respec-
tively, from average- to high-risk categories, whereas the 2.9%
and 3.3% of individuals was inappropriately reclassified from
average- to low-risk and from high- to average-risk categories.
Conversely, among individuals who did not develop sICH, the
STARTING-SICH resulted in the 9% and 11.8% of individu-
als being reclassified, respectively, from average- to low-risk
and from high- to average-risk categories, whereas the 1.8%
of individuals was inappropriately reclassified from average-
to high-risk categories. No significant difference was observed
when we used the risk categories <3%, 3% to 6%, and >6%.
The AUC-ROC of the STARTING-SICH nomogram for
predicting the probability of SITS-MOST sICH was 0.683
(95% CI, 0.655–0.728) in the training set and 0.721 (95% CI,
0.662–0.779) in the test set.
Discussion
We present here the STARTING-SICH nomogram, which pre-
dicts the probability of sICH per the ECASS-II definition for
individual stroke patients receiving intravenous thrombolysis.
The STARTING-SICH nomogram is a graphical calcula-
tion instrument that allows the use of predictive continuous
variables in their maximum interval amplitude, bypassing
the use of the statistical expedient of the artificial cutoff at
the noncategorical to separate the patients into ≥2 different
clusters. By converting the total score into a continuum of
individual probability, the STARTING-SICH nomogram
reclassifies better the ECASS-II sICH risk from current low-
to average-risk and from average- to high-risk categories, and
 Cappellari et al   Nomogram to Predict sICH After Thrombolysis   401

Figure 2. STARTING-SICH (systolic blood pressure, age, onset-to-treatment time for thrombolysis, NIHSS score, glucose, aspirin alone,
aspirin plus clopidogrel, anticoagulant with INR ≤1.7, current infarction sign, hyperdense artery sign) nomogram calibration plot. Calibra-
tion plot displaying the observed proportion of patients who developed ECASS-II (European Cooperative Acute Stroke Study-II) symp-
tomatic intracerebral hemorrhage (sICH) in the test cohort (dots) with 95% confidence intervals (vertical lines) against STARTING-SICH
total score values; the solid curve shows the predicted values based on the STARTING-SICH nomogram.
Downloaded from http://ahajournals.org by on September 29, 2023

vice versa, compared with SITS-SICH and SEDAN scores.
However, different statistical performances are subordinated
to the choice of risk thresholds, and these remain controver-
sial. The present study is unable to show a significant superi-
ority of the nomogram for prediction of an ECASS-II sICH
probability >10% (ie, STARTING-SICH total score >27)
because of the low number of patients (ie, ≈1%) with a high
score. However, the recent revision of guidelines indicating
intravenous thrombolysis <4.5 hours of stroke onset without
upper limit of age and NIHSS score will increase the limit of
current scores based on risk-grouping categorization of age
and NIHSS score,12 because the proportion of elderly patients
and severe strokes undergoing intravenous thrombolysis will
expectedly increase. In patients >80 years of age, the risk of
sICH could be increased by the increased prevalence of cere-
bral small-vessel disease (ie, hypertensive arteriopathy and
cerebral amyloid angiopathy) and comorbidities that require
the use of treatment with antiplatelets and anticoagulants. In
patients with NIHSS score >25, the risk of ECASS-II sICH
could be increased especially in anterior infarcts (≈2/3 of
severe strokes).18 Also, the prevalence of hyperglycemia could
increase in relation to brain infarction severity, even if a linear
relationship between glucose levels and NIHSS score has not
been observed.19 On the other side, the STARTING-SICH total
score was explored up to 31 points by using the entire Italian
SITS-ISTR cohort, whereas the other 2 scores were explored
up to the last point, although age >80 years and NIHSS score
>25 were off-label conditions observed only in the 13.8%
and 0.9% of patients, respectively. A future potentially high-
risk population of elderly patients and severe strokes will be
needed to test the predictive accuracy of the STARTING-
SICH nomogram. However, we used a model that included
the continuous predictors in their maximum interval ampli-
tude according to the new eligibility criteria for intravenous
thrombolysis. Therefore, the nomogram could be favored over
previous prognostic scores based on outdated risk-grouping
categorization for identifying different risk categories in
patients subjected to new potential risk combinations.
The ECASS-II sICH rate was found to be lower in our
study than the entire SITS-ISTR cohort (3.6% versus 4.5%).
This was observed despite the Italian cohort is the first con-
tributor to the SITS-ISTR database and the characteristics
of our patients are comparable to those of the entire SITS-
ISTR cohort (http://www.sitsinternational.org/registries/sits-
thrombolysis). Because sICH defined according to ECASS-II
criteria is the one with the highest interrater agreement,20 the
complexity of the underlying pathophysiological mechanisms
may explain the variable incidence of sICH.
Our nomogram is composed by 10 clinical predictors of
sICH that are readily available before intravenous throm-
bolysis. Blood glucose, NIHSS score, age, systolic BP, and
onset-to-treatment time for thrombolysis are the noncategor-
ical variables by order of decreasing multivariate predictive
effect. Compared with the cutoff values for the continuous
variables used by SITS-SICH and SEDAN scores, several
different risk categorizations were integrated into other
 402  Stroke  February 2018

Figure 3. Proportion of ECASS-II (European Cooperative Acute Stroke Study-II) symptomatic intracerebral hemorrhage (sICH) per increas-
ing points of the STARTING-SICH (systolic blood pressure, age, onset-to-treatment time for thrombolysis, NIHSS score, glucose, aspirin
alone, aspirin plus clopidogrel, anticoagulant with INR ≤1.7, current infarction sign, hyperdense artery sign) total score. Bars show the rate
of sICH for each score point. Percentages indicate rates per score point. At risk: proportion of all patients with the respective risk score.

points-based risk scores for sICH after thrombolysis. Blood
glucose <100, 100 to 149, and ≥150 mg/dL, NIHSS score 0
to 5, 6 to 10, 11 to 15, 16 to 20, and >20, age ≤60, 61 to 70,
and 71 to 80 years, systolic BP <120, 120 to 149, 150 to 179,
Downloaded from http://ahajournals.org by on September 29, 2023
and ≥180 mm Hg were the risk categories included in the
GRASPS (glucose, race, age, sex, systolic BP, and NIHSS
score).21 Blood glucose >200 mg/dL, NIHSS score 15 to
20 and ≥20 were the risk categories incorporated into the

Figure 4. The discriminative performance of the STARTING-SICH (systolic blood pressure, age, onset-to-treatment time for thrombolysis,
NIHSS score, glucose, aspirin alone, aspirin plus clopidogrel, anticoagulant with INR ≤1.7, current infarction sign, hyperdense artery sign)
nomogram, SITS (Safe Implementation of Thrombolysis in Stroke)-SICH score, and SEDAN (blood sugar, early infarct signs and hyper-
dense cerebral artery sign, age, and NIHSS) score. NIHSS indicates National Institutes of Health Stroke Scale.
 Cappellari et al   Nomogram to Predict sICH After Thrombolysis   403
HAT score (Hemorrhage After Thrombolysis).22 Blood glu-
cose >150 mg/dL, NIHSS score ≥11, age ≥60 years were the
risk categories integrated into the MMS score (Multicenter
Stroke Survey).23 Despite the apparent rationality, the pro-
cess of categorizing discrete/continuous variables into 2 or
3 risk groups, as used by SITS-SICH and SEDAN scores, is
often statistically inefficient and may decrease the predic-
tive accuracy. The first problem caused by the categorization
of a continuous variable is the loss of information; that loss
is small with several groups and is most severe with just 2
groups. A further important disadvantage of dichotomization
is that it does not make use of within-category information.
Everyone above or below the cut point is treated as equal, yet
the prognosis may vary considerably. Individuals close to but
on opposite sides of the cut point are characterized as being
very different rather than very similar.
The strongest categorical predictor of sICH in our model is
dual antiplatelet therapy with aspirin and clopidogrel as previ-
ously reported.24,25 The relationship between points assigned
to aspirin plus clopidogrel and aspirin alone on the prelimi-
nary score of the nomogram (ie, 1.93) was similar to the
SITS-SICH score, as well as the relationship between points
assigned to current infarction sign and hyperdense artery sign
on the nomogram (ie, 1.11) was similar to the SEDAN score.
In addition, the STARTING-SICH nomogram incorporates
oral anticoagulant with INR ≤1.7. Despite the data being
uncertain,26–28 the true absolute risk of sICH in this population
remains a matter of debate because warfarin-treated patients
were excluded from major trials of intravenous thrombolysis.
Downloaded from http://ahajournals.org by on September 29, 2023
Our study has some limitations. First, it is based on a ret-
rospective analysis of an ongoing database, so bearing the
limitations of such study design. Despite our belief that the
patient data in the entire Italian SITS-ISTR cohort are repre-
sentative of clinical practice across a variety of demographics
and stroke center types, for the risk score to be suitable for
routine clinical practice, an external validation in a completely
different cohort is warranted. Second, the number of missing
data for patients receiving oral anticoagulants with known
INR and for determining sICH per the ECASS-II definition
may have influenced the final outcome. This is a limitation
of all studies that used a large stroke thrombolysis database
such as the SITS-ISTR, including the studies that validated
the SITS-SICH and SEDAN scores.1,2 Third, data on platelet
count, infarct size at baseline, and ethnicity are lacking in the
SITS-ISTR. For this reason, we have been unable to compare
the performances of our nomogram with GRASPS,21 HAT,22
and MSS23 scores; however, they have not been validated in
a large stroke thrombolysis database such as the SITS-ISTR.
Fourth, data on the Alberta Stroke Program Early CT Score
(ASPECTS) and Fazekas score, for continuous degree of
extent of early infarct signs and severity of white matter dis-
ease, respectively, are not available in the SITS-ISTR. Finally,
biomarkers such as cerebral microbleeds are not included
into the model because magnetic resonance imaging is not
performed routinely before thrombolysis. Future prospec-
tive studies will have to assess whether the integration of
novel biomarkers may help to improve the accuracy of the
STARTING-SICH nomogram prediction. Notwithstanding
these limitations, to our knowledge, the present study is the
first attempt to develop and validate a nomogram to predict the
probability of sICH in a large cohort of stroke patients treated
with intravenous thrombolysis in agreement with the current
guidelines.
Conclusions
The STARTING-SICH nomogram was developed and inter-
nally validated in a large SITS-ISTR cohort to predict the
probability of sICH per ECASS-II definition after intrave-
nous thrombolysis for stroke. Although there are no sub-
stantial differences of discriminative performances between
the models, the STARTING-SICH nomogram may be a new
means more reliable than SITS-SICH and SEDAN scores for
estimating risk of sICH because it is based on a combina-
tion of individual clinical characteristics of the patient. Our
nomogram may be easily and quickly applicable in the clini-
cal setting if used on a computer or a handheld device with
the related software.
Acknowledgments
We thank all the Italian centers registering patients in the SITS-ISTR
and their investigators for their participation.
Disclosures
Dr Toni declares to have speaking engagements and a consultant
or advisory relationship with Boehringer Ingelheim, Bayer, Pfizer,
Bristol Meyer Squibb, and Daiichi Sankyo. The other authors report
no conflicts.
References
1. Mazya M, Egido JA, Ford GA, Lees KR, Mikulik R, Toni D, et al;
SITS Investigators. Predicting the risk of symptomatic intracerebral
hemorrhage in ischemic stroke treated with intravenous alteplase: safe
Implementation of Treatments in Stroke (SITS) symptomatic intracere-
bral hemorrhage risk score. Stroke. 2012;43:1524–1531. doi: 10.1161/
STROKEAHA.111.644815.
2. Mazya MV, Bovi P, Castillo J, Jatuzis D, Kobayashi A, Wahlgren N, et al.
External validation of the SEDAN score for prediction of intracerebral
hemorrhage in stroke thrombolysis. Stroke. 2013;44:1595–1600. doi:
10.1161/STROKEAHA.113.000794.
3. Álvarez-Sabín J, Maisterra O, Santamarina E, Kase CS. Factors influ-
encing haemorrhagic transformation in ischaemic stroke. Lancet Neurol.
2013;12:689–705. doi: 10.1016/S1474-4422(13)70055-3.
4. Shariat SF, Capitanio U, Jeldres C, Karakiewicz PI. Can nomograms be
superior to other prediction tools? BJU Int. 2009;103:492–495; discus-
sion 495. doi: 10.1111/j.1464-410X.2008.08073.x.
5. Chun FK, Karakiewicz PI, Briganti A, Walz J, Kattan MW, Huland H,
et al. A critical appraisal of logistic regression-based nomograms, artifi-
cial neural networks, classification and regression-tree models, look-up
tables and risk-group stratification models for prostate cancer. BJU Int.
2007;99:794–800. doi: 10.1111/j.1464-410X.2006.06694.x.
6. Kattan MW. Nomograms are superior to staging and risk grouping
systems for identifying high-risk patients: preoperative application in
prostate cancer. Curr Opin Urol. 2003;13:111–116. doi: 10.1097/01.
mou.0000058631.64616.54.
7. Kattan MW. Comparison of Cox regression with other methods for
determining prediction models and nomograms. J Urol. 2003;170(6 pt
2):S6–S9; discussion S10. doi: 10.1097/01.ju.0000094764.56269.2d.
8. Shariat SF, Karakiewicz PI, Palapattu GS, Amiel GE, Lotan Y, Rogers
CG, et al. Nomograms provide improved accuracy for predicting sur-
vival after radical cystectomy. Clin Cancer Res. 2006;12:6663–6676.
doi: 10.1158/1078-0432.CCR-06-0372.
9. Jehi L, Yardi R, Chagin K, Tassi L, Russo GL, Worrell G, et al.
Development and validation of nomograms to provide individual-
ised predictions of seizure outcomes after epilepsy surgery: a retro-
spective analysis. Lancet Neurol. 2015;14:283–290. doi: 10.1016/
S1474-4422(14)70325-4.
 404  Stroke  February 2018
10. Callegaro D, Miceli R, Bonvalot S, Ferguson P, Strauss DC, Levy A,
et al. Development and external validation of two nomograms to pre-
dict overall survival and occurrence of distant metastases in adults after
surgical resection of localised soft-tissue sarcomas of the extremities:
a retrospective analysis. Lancet Oncol. 2016;17:671–680. doi: 10.1016/
S1470-2045(16)00010-3.
11. Hijazi Z, Oldgren J, Lindbäck J, Alexander JH, Connolly SJ, Eikelboom
JW, et al; ARISTOTLE and RE-LY Investigators. The novel biomarker-
based ABC (age, biomarkers, clinical history)-bleeding risk score for
patients with atrial fibrillation: a derivation and validation study. Lancet.
2016;387:2302–2311. doi: 10.1016/S0140-6736(16)00741-8.
12. Toni D, Mangiafico S, Agostoni E, Bergui M, Cerrato P, Ciccone A, et al.
Intravenous thrombolysis and intra-arterial interventions in acute isch-
emic stroke: Italian Stroke Organisation (ISO)-SPREAD guidelines. Int
J Stroke. 2015;10:1119–1129. doi: 10.1111/ijs.12604.
13. Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D,
et al. Randomised double-blind placebo-controlled trial of thrombolytic
therapy with intravenous alteplase in acute ischaemic stroke (ECASS
II). Second European-Australasian Acute Stroke Study Investigators.
Lancet. 1998;352:1245–1251.
14. Wahlgren N, Ahmed N, Dávalos A, Ford GA, Grond M, Hacke W, et
al; SITS-MOST Investigators. Thrombolysis with alteplase for acute
ischaemic stroke in the Safe Implementation of Thrombolysis in Stroke-
Monitoring Study (SITS-MOST): an observational study. Lancet.
2007;369:275–282. doi: 10.1016/S0140-6736(07)60149-4.
15. Strbian D, Engelter S, Michel P, Meretoja A, Sekoranja L, Ahlhelm FJ, et
al. Symptomatic intracranial hemorrhage after stroke thrombolysis: the
SEDAN score. Ann Neurol. 2012;71:634–641. doi: 10.1002/ana.23546.
16. Cook NR, Ridker PM. Advances in measuring the effect of individual
predictors of cardiovascular risk: the role of reclassification measures.
Ann Intern Med. 2009;150:795–802.
17. Schnabel RB, Larson MG, Yamamoto JF, Sullivan LM, Pencina MJ,
Meigs JB, et al. Relations of biomarkers of distinct pathophysiological
pathways and atrial fibrillation incidence in the community. Circulation.
2010;121:200–207. doi: 10.1161/CIRCULATIONAHA.109.882241.
18. Mazya MV, Lees KR, Collas D, Rand VM, Mikulik R, Toni D, et al.
IV thrombolysis in very severe and severe ischemic stroke: results from
Downloaded from http://ahajournals.org by on September 29, 2023
the SITS-ISTR Registry. Neurology. 2015;85:2098–2106. doi: 10.1212/
WNL.0000000000002199.
19. Desilles JP, Meseguer E, Labreuche J, Lapergue B, Sirimarco G,
Gonzalez-Valcarcel J, et al. Diabetes mellitus, admission glucose, and
outcomes after stroke thrombolysis: a registry and systematic review.
Stroke. 2013;44:1915–1923. doi: 10.1161/STROKEAHA.111.000813.
20. Gumbinger C, Gruschka P, Böttinger M, Heerlein K, Barrows R,
Hacke W, et al. Improved prediction of poor outcome after thromboly-
sis using conservative definitions of symptomatic hemorrhage. Stroke.
2012;43:240–242. doi: 10.1161/STROKEAHA.111.623033.
21. Menon BK, Saver JL, Prabhakaran S, Reeves M, Liang L, Olson DM, et
al. Risk score for intracranial hemorrhage in patients with acute ischemic
stroke treated with intravenous tissue-type plasminogen activator. Stroke.
2012;43:2293–2299. doi: 10.1161/STROKEAHA.112.660415.
22. Lou M, Safdar A, Mehdiratta M, Kumar S, Schlaug G, Caplan L, et
al. The HAT score: a simple grading scale for predicting hemorrhage
after thrombolysis. Neurology. 2008;71:1417–1423. doi: 10.1212/01.
wnl.0000330297.58334.dd.
23. Cucchiara B, Tanne D, Levine SR, Demchuk AM, Kasner S. A risk
score to predict intracranial hemorrhage after recombinant tissue plas-
minogen activator for acute ischemic stroke. J Stroke Cerebrovasc Dis.
2008;17:331–333. doi: 10.1016/j.jstrokecerebrovasdis.2008.03.012.
24. Diedler J, Ahmed N, Sykora M, Uyttenboogaart M, Overgaard K,
Luijckx GJ, et al. Safety of intravenous thrombolysis for acute ischemic
stroke in patients receiving antiplatelet therapy at stroke onset. Stroke.
2010;41:288–294. doi: 10.1161/STROKEAHA.109.559724.
25. Cucchiara B, Kasner SE, Tanne D, Levine SR, Demchuk A, Messe SR,
et al; SAINT Investigators. Factors associated with intracerebral hemor-
rhage after thrombolytic therapy for ischemic stroke: pooled analysis of
placebo data from the Stroke-Acute Ischemic NXY Treatment (SAINT)
I and SAINT II Trials. Stroke. 2009;40:3067–3072. doi: 10.1161/
STROKEAHA.109.554386.
26. Ruecker M, Matosevic B, Willeit P, Kirchmayr M, Zangerle A, Knoflach
M, et al. Subtherapeutic warfarin therapy entails an increased bleeding
risk after stroke thrombolysis. Neurology. 2012;79:31–38. doi: 10.1212/
WNL.0b013e31825dcdf0.
27. Miedema I, Luijckx GJ, De Keyser J, Koch M, Uyttenboogaart M.
Thrombolytic therapy for ischaemic stroke in patients using warfarin:
a systematic review and meta-analysis. J Neurol Neurosurg Psychiatry.
2012;83:537–540. doi: 10.1136/jnnp-2011-301794.
28. Xian Y, Liang L, Smith EE, Schwamm LH, Reeves MJ, Olson DM, et
al. Risks of intracranial hemorrhage among patients with acute isch-
emic stroke receiving warfarin and treated with intravenous tissue
plasminogen activator. JAMA. 2012;307:2600–2608. doi: 10.1001/
jama.2012.6756.