European Heart Journal (2024) 45, 129–131 EDITORIAL

https://doi.org/10.1093/eurheartj/ehad748 Epidemiology, prevention, and health care policies

Adapting cardiovascular risk prediction

Downloaded from https://academic.oup.com/eurheartj/article/45/2/129/7427524 by National Science & Technology Library user on 30 January 2024
models to different populations: the need
for recalibration
Lisa Pennells 1,2, Stephen Kaptoge 1,2,
and Emanuele Di Angelantonio 1,2,3,4,5,6*
1
BHF Cardiovascular Epidemiology Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge CB1 8RN, UK; 2Victor Phillip Dahdaleh Heart and Lung
Research Institute, University of Cambridge, Cambridge, UK; 3BHF Centre of Research Excellence, School of Clinical Medicine, Addenbrooke’s Hospital, University of Cambridge,
Cambridge, UK; 4Health Data Research UK Cambridge, Wellcome Genome Campus and University of Cambridge, Hinxton, UK; 5NIHR Blood and Transplant Research Unit in
Donor Health and Behaviour, University of Cambridge, Cambridge, UK; and 6Health Data Science Centre, Human Technopole, Milan, Italy

Online publish-ahead-of-print 18 November 2023

This editorial refers to ‘Recommendations for statin management in primary prevention: disparities among international
risk scores’, by G.B.J. Mancini et al., https://doi.org/10.1093/eurheartj/ehad539.

Graphical Abstract

before and after recalibration

FRS vs. SCORE PCE vs. SCORE RRS vs. SCORE

Number at high risk N = 36 794 N = 39 294 N = 34 122

14% 14%
0.1%
FRS >7.5%
Before SCORE >5%
recalibration 66% 75% 79% PCE >7.5%
RRS >7.5%

20% 25% 7%

Number at high risk N = 24 708 N = 25 151 N = 24 689

12% 3% 11%
FRS >7.5%
After recalibration SCORE >7.5%
to common CVD
76% 84% 80% PCE >7.5%
endpoint
RRS >7.5%

12% 13% 9%

Pairwise overlap in individuals classed as high risk according to four different CVD risk algorithms, before and after recalibration. The figure is from a
previously published EHJ paper which compared the performance of four risk scores using data from 86 cohort studies involved in the Emerging Risk
Factors Collaboration.1 FRS, Framingham Risk Score; SCORE, Systematic Coronary Risk Evaluation (SCORE) risk models; PCE, Pooled Cohort
Equations; RRS, Reynolds Risk Score.

The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal or of the European Society of Cardiology.
* Corresponding author. Tel: +44 1223 740522, Fax: +44 1223 740329, Email: ed303@medschl.cam.ac.uk
© The Author(s) 2023. Published by Oxford University Press on behalf of the European Society of Cardiology. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
130
The research presented by John Mancini and co-authors in this issue
of the European Heart Journal shows an interesting comparison of car­
diovascular disease (CVD) risk classifications using alternative US- and
European-based 10-year CVD risk prediction models together with
guideline-recommended risk categorizations.2 Four risk prediction
models are compared; (i) the Framingham Risk Score (FRS); (ii) the
Pooled Cohort Equation (PCE); (iii) the updated Systematic
Coronary Risk Evaluation 2 algorithm (SCORE2); and (iv) the
Multi-Ethnic Study of Atherosclerosis (MESA) Risk Algorithm which
incorporates the coronary artery calcium score (CACS). The risk
models are compared in a simulated dataset, which the authors
declare could represent ‘any part of the world’. The main finding
on application of the different risk prediction models to this
hypothetical population is that individuals with the same risk factor
profile get very different risk estimates according to the algorithm
used. Furthermore, when considering guideline-recommended risk ca­
tegorizations, very different proportions of individuals would be con­
sidered as intermediate risk, or high risk and eligible for lipid-lowering
treatment.
Mancini et al. take what they refer to as a novel approach to compari­
son of the risk scores, applying each prediction model to a hypothetical
cohort that is simulated by forming all possible permutations and com­
binations of a fixed set of risk factor values to yield a dataset of 7680
theoretical patients. However, a key limitation of the approach is that
it does not reflect realistic risk factor distributions or correlations be­
tween risk factors. Equal numbers of men and women, smokers and
non-smokers, are generated, and no weighting is applied to risk factor
combinations more likely to be observed in real primary prevention po­
pulations. This approach yields a dataset that is unlikely to be observed
in any part of the world. Even the sensitivity analysis that used risk factor
distributions similar to those observed in NHANES unrealistically as­
sumes an absence of correlation between risk factors. It may be true
that there is discordance between risk classification according to the
risk prediction models and guidelines, but the degree to which this oc­
curs might be very different using more realistic data. In fact, previous
work published in the European Heart Journal, using observed data
from >80 different cohort studies, showed much greater alignment
of categorization using common risk prediction models (Graphical
Abstract).1
That aside, the lack of concordance between risk categorization is
considered by the authors to be a limitation of the current risk predic­
tion models and their recommend approaches to treatment. This con­
clusion is based on the assumption that someone with a certain risk
factor profile should receive the same recommendation regardless of
where they live in the world. This assumption would be sensible if
the risk factors included in CVD risk prediction models explain 100%
of the variation in CVD risk in a population. Sadly they do not.
Previous work has shown that risk factors alone explained ∼30% of
the variation in CVD mortality risk between distinct populations,3
and the population fraction of 10-year CVD incidence attributable to
the combined effects of five major modifiable CVD risk factors (i.e.
body mass index, non-HDL-cholesterol, systolic blood pressure, smok­
ing, and diabetes) was recently estimated to be <60% across multiple
populations.4 Many undiscovered and unmeasurable factors explain
the remaining variation in CVD incidence and there are large differ­
ences in population CVD risk across countries and even regions within
countries, findings not explained by the conventional CVD risk factors.
The SCORE2 algorithms5,6 attempt to account for these large differ­
ences in population risks by providing four versions that are calibrated
to the background level of risk in different regions of Europe. This is a
Editorial
somewhat crude way of accounting for unexplained variation in risk
but results in more accurate risk estimates on average for each
population.
The authors also assume that differences in risks assigned by different
algorithms to individuals with the same risk factor profile are due to dif­
ferent weightings given to each risk factor. This is in fact not true; actu­
ally the weighting applied to each risk factor [known as a hazard ratio
Downloaded from https://academic.oup.com/eurheartj/article/45/2/129/7427524 by National Science & Technology Library user on 30 January 2024
(HR)] is very similar across risk prediction models, reflecting homogen­
eity in relative risk factor effects across the different populations from
which the algorithms are derived.7 The similar weighting of risk factors
is evident in the high correlations observed between different risk pre­
diction models of the order of .78–1.00 and the very similar ability to
discriminate incident CVD events as previously found.1 What causes
the variation in absolute risk predictions across risk models is the differ­
ent background level of risk in each dataset from which the study was
derived (i.e. the baseline risk, which the HRs act upon to give an indivi­
dual’s absolute risk). The baseline risk in the Framingham dataset is
much higher than that in the PCE dataset, not least because a larger
set of endpoints are included in the CVD definition, but also because
the participants included in the studies represent different subsets of
the American population from different time periods. Hence, despite
similar weights applied to each risk factor, these are applied to popula­
tions with different baseline population risk, with resultant different risk
estimates (higher for FRS). This fact was also illustrated in previous
work that showed that simple recalibration (i.e. resetting the back­
ground absolute risk predicted by the model) equalized the perform­
ance of four different risk scores, resulting in similar treatment
recommendations for all individuals.1
The study of Mancini et al. also addresses the question of whether or
not the assignment of treatment should be based on absolute risk at all.
In other words, should high-risk individuals be identified based on a
collection of risk factors alone, or on risk estimated using collections
of risk factors combined with the background level of risk characterizing
that population? If the first option is desired, we assume all we need to
know is the combined level of exposure to a number of risk factors; so
we are not really estimating risk at all—rather this yields an arbitrary
score. Decisions then need to be made about what constitutes a high
score. If we genuinely wish to estimate the true risk for an individual,
we should take into account the background level of risk relevant
to the population an individual is from. This is most effectively done
by recalibration of risk scores to contemporary circumstances of target
populations, as recently done with SCORE2. FRS and PCE, in contrast,
assume that baseline risk in the USA is the same as baseline risk in the
datasets used for model derivation. Furthermore, if the background
population level of risk is taken into account, it is appropriate that an
individual should indeed get a different risk estimate for the same risk
factor values, according to where they are from, exactly as observed
by Mancini et al.
In favour of the arbitrary score only approach, the authors argue that
‘it is well recognized that the relative benefits of statins extend even to
lower risk patients’. However, the key part of this phrase is that the
‘relative’ effect of statins on CVD risk is the same even at lower risks.
It therefore follows that the absolute benefit is far less for those at low­
er risk. Assuming that statins reduce risk by 20%,8 this infers a small
benefit to someone whose absolute CVD risk is only 1% (reducing it
to .8%, or prevention of 1 event per 500 similar people treated for
10 years), whereas to someone at 30% risk the benefit is far greater (re­
ducing risk to 24%, akin to 30 fewer CVD events per 500 similar people
treated). If the individual is from a population with extremely low event
rates, using an arbitrary score to assign treatment would lead to
Editorial
treatment of many people who would never otherwise go on to have a
CVD event. Conversely in a very high-risk population, many people
would not be assigned treatment because their risk factors are deemed
‘lower’ in value, yet would go on to have CVD events.
A final factor to take into account is that countries differ in their
infrastructure and funding available to assess and treat individuals
at high CVD risk, and no risk threshold is universally applicable.
Country-specific decisions and recommendations regarding screening
strategies and treatment thresholds are therefore needed. These
need to reflect the level of risk in the population and the likely
proportion of individuals who might be recommended treatment.
Furthermore, the decision to initiate interventions also remains a mat­
ter of individual consideration and shared decision-making. Hence, var­
iations in guideline recommendations across populations should be
expected, and comparison of different guideline-recommended ap­
proaches in a single dataset from one single population may not be par­
ticularly relevant.
Declarations
Disclosure of Interest
All authors declare no disclosure of interest for this contribution.
131
References
1. Pennells L, Kaptoge S, Wood A, Sweeting M, Zhao X, White I, et al. Equalization of four
cardiovascular risk algorithms after systematic recalibration: individual-participant
meta-analysis of 86 prospective studies. Eur Heart J 2019;40:621–31. https://doi.org/10.
1093/eurheartj/ehy653
2. Mancini GBJ, Ryomoto A, Yeoh E, Brunham LR, Hegele RA. Recommendations for statin
management in primary prevention: disparities among international risk scores. Eur Heart J
2024;45:117–28. https://doi.org/10.1093/eurheartj/ehad539
Downloaded from https://academic.oup.com/eurheartj/article/45/2/129/7427524 by National Science & Technology Library user on 30 January 2024
3. Trias-Llimos S, Pennells L, Tverdal A, Kudryavtsev AV, Malyutina S, Hopstock LA, et al.
Quantifying the contribution of established risk factors to cardiovascular mortality differ­
ences between Russia and Norway. Sci Rep 2020;10:20796. https://doi.org/10.1038/
s41598-020-77877-3
4. Global Cardiovascular Risk Consortium. Global effect of modifiable risk factors on car­
diovascular disease and mortality. N Engl J Med 2023;389:1273–85. https://doi.org/10.
1056/NEJMoa2206916. Epub ahead of print. PMID: 37632466.
5. SCORE2-OP working group and ESC Cardiovascular risk collaboration. SCORE2-OP risk pre­
diction algorithms: estimating incident cardiovascular event risk in older persons in four geo­
graphical risk regions. Eur Heart J 2021;42:2455–67. https://doi.org/10.1093/eurheartj/ehab312
6. SCORE2 working group and ESC Cardiovascular risk collaboration. SCORE2 risk predic­
tion algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe.
Eur Heart J 2021;42:2439–54. https://doi.org/10.1093/eurheartj/ehab309
7. WHO CVD Risk Chart Working Group. World Health Organization cardiovascular dis­
ease risk charts: revised models to estimate risk in 21 global regions. Lancet Glob Health
2019;7:e1332–45. https://doi.org/10.1016/S2214-109X(19)30318-3
8. Collins R, Reith C, Emberson J, Armitage J, Baigent C, Blackwell L, et al. Interpretation of
the evidence for the efficacy and safety of statin therapy. Lancet 2016;388:2532–61.
https://doi.org/10.1016/S0140-6736(16)31357-5