Download as pdf or txt
Download as pdf or txt
You are on page 1of 8

Clinical Radiology 78 (2023) 99e106

Contents lists available at ScienceDirect

Clinical Radiology
journal homepage: www.clinicalradiologyonline.net

The top 100 most cited articles on artificial


intelligence in radiology: a bibliometric analysis
H. Hughes a, *, M. O’Reilly b, N. McVeigh a, R. Ryan a
a
Department of Radiology, St Vincent’s University Hospital, Dublin, 4, Ireland
b
Department of Radiology, Cork University Hospital, Wilton, Co. Cork, Ireland

art icl e i nformat ion AIM: To identify the most influential publications relating to artificial intelligence (AI) in
radiology in order to identify current trends in the literature and to highlight areas requiring
Article history: further research.
Received 16 August 2022 MATERIALS AND METHODS: A retrospective bibliometric analysis was performed of the top
Received in revised form 100 most cited articles on this topic. Data pertaining to year of publication, publishing journal,
12 September 2022 journal impact factor, authorship, article title, institution, country, type of article, article sub-
Accepted 16 September 2022 ject, and keywords were collected.
RESULTS: The number of citations per article for the top 100 list ranged from 254 to 3,576
(median 353). The number of citations per year, per article ranged from 10.4 to 894 (median
65.6). The majority of articles (n¼62) were published within the last 10 years. The USA was the
most common country of origin (n¼44). The journal with the greatest number of articles was
IEEE Transactions On Medical Imaging (n¼38). University Medical Center Utrecht contributed
the greatest number of articles (n¼6). There were 92 original research articles, 52 of which
were clinical studies. The most common clinical subjects were neuroimaging (n¼25) and
oncology (n¼16). The most common keyword used was “deep learning” (n¼34).
CONCLUSION: This study provides an in-depth analysis of the top 100 most-cited papers on
the use of AI in radiology. It also provides researchers with detailed insight into the current
influential papers in this field, the characteristics of those studies, as well as potential future
trends in this fast-developing area of radiology.
Ó 2022 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.

Introduction intelligent machines.1 Deep learning is a subtype of ma-


chine learning. Deep-learning algorithms (DLA) are based
The principles supporting the foundation of artificial on the use of artificial neural networks (ANNs) that mimic
intelligence (AI) have existed for a number of decades. Alan neural networks found in the human brain, which process
Turing first described the use of computer technology to information and facilitate communication distribution.2,3
parallel human intelligence and critical analysis in 1950.1 It DLA permit the automatic learning of discriminative fea-
was another 6 years before John McCarthy first coined the tures through complex data interpretation in order to
term “artificial intelligence” in relation to the creation of determine non-linear correlations.4 Turing developed a

* Guarantor and correspondent: H. Hughes, Department of Radiology, St Vincent’s University Hospital, Dublin, 4, Ireland. Tel.: þ353863336385.
E-mail address: h.hughes07@gmail.com (H. Hughes).

https://doi.org/10.1016/j.crad.2022.09.133
0009-9260/Ó 2022 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.
100 H. Hughes et al. / Clinical Radiology 78 (2023) 99e106

means of testing a machine’s capability to perform tasks neighbour” OR “decision tree learning” OR “data mining”
with a degree of intelligence indistinguishable from that of OR “fuzzy” OR “computational intelligence” OR “computer
a human, known as the “Turing Test”.1 It is only in recent reasoning”. Search results were further refined to the sub-
years, however, that machine capabilities have evolved to ject category “radiology, nuclear medicine and medical
such a degree that they hold the computational abilities imaging” on the WoS database. There were no restrictions
required to perform deep-learning tasks similar to or even on year of publication or publication language. Articles were
better than humans.4 included if their title, abstract, or keywords contained the
Experts have highlighted concerns that the advent of AI search terms of interest.
and its incorporation into clinical medicine could lead to the In total, the search returned 11,210 results. The articles
clinician and radiologist becoming an unnecessary link in were ranked in order of citation count, from highest to
the diagnostic chain5. In relation to automation of work- lowest. The 100 articles with the highest citation count were
place functions, it is known that if a number of tasks in a then downloaded to a local database. Two reviewers (H.H.
chain can be automated, the economic value of human and M.O.) analysed each article and collated data pertaining
input for those that are not automated will rise5; however, to journal of publication, year of publication, authorship,
there are a number of human tasks in the day-to-day author institution, country, article type, keywords, article
practice of a radiologist that cannot be automated or subject, and journal impact factor. Author institution and
assigned an economic value, such as clinical judgement and country were derived from author affiliations. Articles were
multidisciplinary team meetings. Ultimately, it is believed screened to extract keywords and classified into original
that AI will serve to support human function in clinical research or review articles. Studies were further subdivided
medicine, rather than surpass or replace it. into clinical or non-clinical categories. Clinical studies were
Bibliometric analysis is the process of evaluating pub- those that included human participants or those related to
lished literature on a particular subject.6 Citation frequency clinical applications of AI. The full-text of each clinical article
is used to ascertain how frequently a publication is cited by was evaluated to determine the predominant focus of the
other researchers.7 It involves the collation of data in order study; this was used to classify the article’s subject category.
to determine the most influential publications in that sub- The 2020 SCImago Journal Rank (Elsevier, Amsterdam,
ject, to identify trends in certain research fields, and to Netherlands) was used to obtain journal impact factors.10 The
determine potential gaps where further research is average number of citations per year for each article (total
required.6 A potential, important limitation of bibliometric number of citations divided by number of years since pub-
analysis, however, is the phenomenon whereby findings lication) was also calculated as an adjunctive measure of
from seminal papers may become incorporated into general article impact, a method previously employed by other bib-
knowledge within that speciality, and therefore, are not liometric studies.6,7
always cited in the more contemporary literature.8 The aim
of this study was to provide clinicians and researchers with
a list of the most influential publications in the field of AI in Results
radiology. In addition, it also provides an overview and
analysis of how trends in this area have evolved over time, The top 100 journal articles relating to the use of AI in
as well as advances in AI in radiology. radiology were ranked in order of citation count. Table 1
lists the characteristics of the top 50 most cited articles in
AI in radiology. The characteristics of the top 100 most cited
Materials and methods articles are listed in Electronic Supplementary Material
Table 1. The article with the highest number of citations
This study was conducted as a retrospective bibliometric was “A survey on deep learning in medical image analysis”
analysis and did not include human or animal subjects. published in Medical Image Analysis in 2017, with 3,576 ci-
Therefore, ethical approval was not required. An analysis of tations.11 When ranked according to citations per year, it
the top 100 most-cited peer-reviewed journal articles per- remained the highest-ranking article, with an average of
taining to the use of AI in the field of radiology was con- 894 citations per year. The median number of citations in
ducted in November 2021. Articles were identified from the the top 100 list was 353 (range 254 to 3,576). The median
Medline (U.S. National Library of Medicine, Bethesda, USA) number of citations per year was 65.6 (range 10.4e894).
database using the Web of Science (WoS, Clarivate Ana- Sixty-two out of the top 100 articles were published
lytics, Philadelphia, USA) citation indexing service. Journal within the last decade (from 2011 onwards). Just nine arti-
impact factors were identified from the Clarivate “Journal cles were published before the year 2000. The earliest
Citation Reports” 2020 database.9 publication in the list was published in 1994 in IEEE Trans-
Journal articles were identified using the following actions On Medical Imaging, entitled “Morphometric anal-
search strategy: “artificial intelligence” OR “machine intel- ysis of white-matter lesions in MR-images, method and
ligence” OR “deep learning” OR “machine learning” OR validation”.12
“natural language processing” OR “support vector machine” In total, 21 journals contributed to the top 100 list. The
OR “naive Bayes” OR “Bayesian learning” OR “artificial journal with the greatest number of articles in the top 100
neural network” OR “neural network” OR “convolutional list was IEEE Transactions On Medical Imaging (impact factor:
neural network” OR “random forest” OR “k-nearest 10.04), with a contribution of 38 publications. This was
H. Hughes et al. / Clinical Radiology 78 (2023) 99e106 101

Table 1
Characteristics of the top 50 most cited articles in artificial intelligence in radiology.

Rank No. cit. Cit. Per year Year pub. Art. Type Journal Article title
1 3,576 894 2017 NC Medical Image A survey on deep learning in medical image analysis
Analysis
2 2,876 575.2 2016 NC Radiology Radiomics: images are more than pictures, they are
data
3 2,060 412 2016 NC IEEE Transactions on Deep convolutional neural networks for computer-
Medical Imaging aided detection: CNN architectures, dataset
characteristics and transfer learning
4 1,517 252.8 2015 C IEEE Transactions on The multimodal brain tumor image segmentation
Medical Imaging benchmark (BRATS)
5 1,416 83.3 2004 NC Neuroimage A hybrid approach to the skull stripping problem in
MRI
6 1,271 317.8 2017 C Medical Image Efficient multi-scale 3D CNN with fully connected CRF
Analysis for accurate brain lesion segmentation
7 1,192 62.7 2002 NC IEEE Transactions on A modified fuzzy C-means algorithm for bias field
Medical Imaging estimation and segmentation of MRI data
8 1,178 69.3 2004 NC IEEE Transactions on Simultaneous truth and performance level estimation
Medical Imaging (STAPLE): an algorithm for the validation of image
segmentation
9 1,152 288 2017 C Medical Image Brain tumor segmentation with deep neural networks
Analysis
10 1,098 219.6 2016 NC IEEE Transactions on Convolutional neural networks for medical image
Medical Imaging analysis: Full training or fine tuning?
11 931 186.2 2016 C IEEE Transactions on Brain tumor segmentation using convolutional neural
Medical Imaging networks in MRI images
12 929 77.4 2009 NC Neuroimage Machine learning classifiers and fMRI: a tutorial
overview
13 908 36.3 1996 NC IEEE Transactions on Adaptive segmentation of MRI data
Medical Imaging
14 869 32.2 1994 C IEEE Transactions on Morphometric analysis of white-matter lesions in MR-
Medical Imaging imagesdmethod and validation
15 845 49.7 2004 NC Academic Radiology Statistical validation of image segmentation quality
based on a spatial overlap index
16 830 55.3 2006 C Neuroimage Reliability in multi-site structural MRI studies: effects
of gradient non-linearity correction on phantom and
human data
17 799 53.3 2006 NC Computerized Medical Fuzzy c-means clustering with spatial information for
Imaging and Graphics image segmentation
18 729 72.9 2011 C Neuroimage Multimodal classification of Alzheimer’s disease and
mild cognitive impairment
19 662 66.2 2011 NC Neuroimage Single-trial analysis and classification of ERP
componentsda tutorial
20 608 60.8 2011 C Neuroimage Automatic classification of patients with Alzheimer’s
disease from structural MRI: a comparison of 10
methods using the ADNI database
21 594 33 2003 NC IEEE Transactions on A shape-based approach to the segmentation of
Medical Imaging medical imagery using level sets
22 592 98.7 2015 NC BMC Medical Imaging Metrics for evaluating 3D medical image
segmentation: analysis, selection, and tool
23 592 26.9 1999 NC IEEE Transactions on Adaptive fuzzy segmentation of magnetic resonance
Medical Imaging images
24 566 141.5 2017 C Radiology Deep learning at chest radiography: automated
classification of pulmonary tuberculosis by using
convolutional neural networks
25 550 110 2016 C IEEE Transactions on Lung pattern classification for interstitial lung diseases
Medical Imaging using a deep convolutional neural network
26 543 135.8 2017 C Human Brain Mapping Deep learning with convolutional neural networks for
EEG decoding and visualization
27 539 107.8 2016 C IEEE Transactions on Pulmonary nodule detection in CT images: false
Medical Imaging positive reduction using multi-view convolutional
networks
28 504 36 2007 NC Neuroimage The non-invasive Berlin brainecomputer interface:
fast acquisition of effective performance in untrained
subjects
29 499 166.3 2018 NC Insights Into Imaging Convolutional neural networks: an overview and
application in radiology
(continued on next page)
102 H. Hughes et al. / Clinical Radiology 78 (2023) 99e106

Table 1 (continued )

Rank No. cit. Cit. Per year Year pub. Art. Type Journal Article title
30 484 30.3 2005 NC Neuroimage Classifying brain states and determining the
discriminating activation patterns: support vector
machine on functional MRI data
31 468 117 2017 NC IEEE Transactions on Low-dose CT with a residual encoderedecoder
Medical Imaging convolutional neural network
32 457 457 2020 C Physical and COVID-19: automatic detection from X-ray images
Engineering Sciences utilizing transfer learning with convolutional neural
in Medicine networks
33 447 149 2018 C IEEE Transactions on H-DenseUNet: hybrid densely connected UNet for liver
Medical Imaging and tumor segmentation from CT volumes
34 445 74.2 2015 C Neuroimage Deep convolutional neural networks for multi-
modality isointense infant brain image segmentation
35 433 86.6 2016 C IEEE Transactions on Automatic segmentation of MR brain images with a
Medical Imaging convolutional neural network
36 433 108.3 2017 NC RadioGraphics Machine learning for medical imaging
37 427 142.3 2018 NC Magnetic Resonance Learning a variational network for reconstruction of
in Medicine accelerated MRI data
38 422 105.5 2017 C Medical Image Large scale deep learning for computer aided detection
Analysis of mammographic lesions
39 397 132.3 2018 NC IEEE Transactions on Low-dose CT image denoising using a generative
Medical Imaging adversarial network with Wasserstein distance and
perceptual loss
40 395 35.9 2010 NC Neuroimage Neurophysiological predictor of SMR-based BCI
performance
41 389 25.9 2006 NC IEEE Transactions on Generalized overlap measures for evaluation and
Medical Imaging validation in medical image analysis
42 383 54.7 2014 C Neuroimage Hierarchical feature representation and multimodal
fusion with deep learning for AD/MCI diagnosis
43 381 31.8 2009 C Magnetic Resonance Classification of brain tumor type and grade using MRI
in Medicine texture and shape in a machine learning scheme
44 380 27.1 2007 NC Neuroimage Temporal classification of multichannel near-infrared
spectroscopy signals of motor imagery for developing
a brain-computer interface
45 375 187.5 2019 NC Zeitschrift Fur An overview of deep learning in medical imaging
Medizinische Physik focusing on MRI
46 364 19.2 2002 C IEEE Transactions on A support vector machine approach for detection of
Medical Imaging microcalcifications
47 363 40.3 2012 C Neuroimage Multi-modal multi-task learning for joint prediction of
multiple regression and classification variables in
Alzheimer’s disease
48 354 118 2018 C IEEE Transactions on A deep cascade of convolutional neural networks for
Medical Imaging dynamic MR image reconstruction
49 354 27.2 2008 NC Neuroimage Combining multivariate voxel selection and support
vector machines for mapping and classification of fMRI
spatial patterns
50 352 88 2017 C Neuroimage Single subject prediction of brain disorders in
neuroimaging: Promises and pitfalls

Cit, citations; pub, published; Art, article; C, clinical; NC, non-clinical; IEEE, Institute of Electrical and Electronics Engineers; CNN, convoluted neural networks;
MRI, magnetic resonance imaging; 3D, three dimensional; CRF, conditional random field; fMRI, functional magnetic resonance imaging; MR, magnetic reso-
nance; ERP, event related potentials; ADNI, Alzheimer’s Disease Neuroimaging Initiative; BMC, BioMed Central; EEG, electroencephalogram; CT, computed
tomography; COVID-19, coronavirus 2019; SMR, sensory motor rhythm; BCI, brainecomputer interface; AD, Alzheimer’s disease; MCI, mild cognitive
impairment; FCNN, fully convolutional neural networks.

followed by Neuroimage (impact factor: 6.56), which College London (n¼5), Radboud University Medical Centre
contributed 27 articles and Medical Image Analysis (impact (n¼4), and University of Pennsylvania (n¼4). A further three
factor: 6.56), which contributed 11. Fourteen journals institutions contributed three articles each, while a further
contributed one article each to the list. The journal with the six contributed two articles each. The USA was the leading
highest impact factor was Radiology (impact factor: 11.1), publication country of origin (n¼44), followed by The
followed by Journal of Nuclear Medicine (impact factor: Netherlands (n¼11), China (n¼9) and the UK (n¼8).
10.06) and IEEE Transactions on Medical Imaging (impact Of the top 100 publications, eight were review articles
factor: 10.04). and 92 were original research articles. Fifty-two of the top
Overall, there were 69 different institutions represented 100 articles were categorised as clinical studies. Thirty-four
in the top 100 list. University Medical Center Utrecht of those studies were published within the last 10 years. The
contributed the greatest number of articles (n¼6), followed most common clinical research subject was neuroimaging
by Massachusetts Institute of Technology (n¼5), Imperial (n¼25), followed by oncology (n¼16), respiratory (n¼7),
H. Hughes et al. / Clinical Radiology 78 (2023) 99e106 103

Figure 1 Analysis of clinical studies: number of articles per subject.

and cardiac (n¼4; Fig 1). Publications relating to the use of per year (n¼255.1), followed by computer science and AI
AI in respiratory imaging had the highest average number of (n¼191.4) and radiology, nuclear medicine and medical
citations per year (n¼136.8) (Fig 2). Two of these articles imaging (n¼123.9).
were on the topic of coronavirus-19 (COVID-19). The clinical The most common keywords used were “deep learning”
research article with the highest number of citations (n¼34), “convolutional neural networks” (n¼21), and
(n¼1,517) was “The multimodal Brain tumor image seg- “magnetic resonance imaging” or “MRI” (n¼21; Table 2).
mentation benchmark (BRATS)”, published in 2015 in IEEE Only one article used “artificial intelligence” as a keyword.
Transactions On Medical Imaging.13 All of the top 100 studies were published in English.
The most common WoS subject category in the top 100
list was computer science and interdisciplinary applications Discussion
(n¼38), followed by neurosciences and neuroimaging
(n¼28), radiology, nuclear medicine, and medical imaging This bibliometric analysis provides an interesting insight
(n¼19), computer science, and AI (n¼11) and biomedical into the use of AI in radiology, both from a clinical and non-
engineering (n¼2). Publications pertaining to the subject of clinical perspective. It highlights the many institutions,
biomedical engineering had the highest average citations journals, and subject areas that have contributed the most

Figure 2 Analysis of clinical studies: average citations per year for each subject.
104 H. Hughes et al. / Clinical Radiology 78 (2023) 99e106

Table 2 whilst mitigating the unwanted effects of inter-interpreter


Most common keywords. variability.4 Studies have demonstrated that the perfor-
Keywords No. Of times used mance of DLA compared to radiologists is similar for both
Deep learning 34 detection and segmentation in magnetic resonance imaging
Convolutional neural networks 21 (MRI) tasks.19 It has been shown that DLA can be used to
Magnetic resonance imaging/MRI 21 imitate the practice of trained radiologists in terms of image
Machine learning 10
interpretation and determining the significance of varia-
Computer aided diagnosis 7
Computer aided detection 5 tions in characteristics in order to formulate a clinical de-
Computed tomography 5 cision4; however, there are some human tasks that simply
Brain-computer interface 3 cannot be automated, such as years of accumulated clinical
Radiology 3 acumen, nuances of clinical judgement, and multidisci-
Artificial intelligence 1
plinary team meetings.
Although AI has shown promise in the realm of diag-
nostic radiology, significant advances have also been made
to this field, as well as how these contributions have in relation to interventional radiology (IR), most notably
developed over time. interventional oncology. The use of AI in this field may
Healthcare is struggling to keep up with technological facilitate IR specialists in guiding patient treatment path-
advances and patient expectations in relation to service ways in the direction of treatment modalities that may
provision, whilst being compared to the quality of service benefit them. Studies have demonstrated the ability of AI to
delivery by international corporations such as Amazon and stratify patients with hepatocellular carcinoma into re-
Apple.14 There has been a significant surge in attempts to sponders and non-responders to chemo-embolisation
incorporate AI into clinical medicine in order to maximise based on MRI data.20 In relation to cerebrovascular events
efficiency of healthcare delivery and improve patient out- and cerebral arteriovenous malformations, AI has also been
comes.15 Healthcare institutions have recognised the po- shown to predict outcomes reliably for patients undergoing
tential benefits that AI can provide.15 Analysis by Accenture procedures for these conditions.21,22 Although AI has the
estimates that AI will cut yearly healthcare costs in the USA ability to guide treatment and predict outcomes, it may not
by up to USD150 billion by 2026.15 Research studies inves- always predict disease origins or the natural history of a
tigating the clinical application of AI in radiology repre- particular pathology. Robust clinical studies are required
sented over half of the top 100 publications. Interestingly, therefore, in the form of randomised control trials, in order
over half of those clinical articles were published in the last to develop standards of care and guide treatment modal-
10 years. This is unsurprising given the ever-increasing ities, thereby marrying AI in radiology and clinical medicine
demand for healthcare provision worldwide, coupled with in a harmonious and safe way.23
a shortage of physicians in many countries.15 Demand for Although the majority of clinical studies were on the
diagnostic radiology provision is growing at a dispropor- topic of neuroimaging, the average number of citations per
tionate rate when compared to the number of available year for articles relating to respiratory imaging was more
trained radiologists.16 This has resulted in an increased need than double that of neuroimaging. Two out of seven pub-
for productivity as a means of compensating, which has lications on the subject of respiratory imaging focused on
greatly contributed to the expected output of radiologists.16 the topic of COVID-19, both of which were published in
Studies have estimated that, on average, in an 8-h workday, 2020. The COVID-19 pandemic saw an unprecedented in-
a radiologist is required to interpret one image every 3e4 crease in the number of publications worldwide; in 2020,
seconds.17 The benefits of finding ways to seamlessly and 4% of published literature was on the topic of COVID-19
safely integrate AI into clinical radiological workflow in alone, with a significant increase seen in the publication
order to maximise efficiency, minimise errors, and reduce of studies on all subjects that same year.24 Notable advances
manual input cannot be underestimated. Efforts in this area have been seen in the adaptation of AI into clinical respi-
have evidently dominated literature output in the past 10 ratory imaging both before and during the COVID-19
years, as is evident in this top 100 list. pandemic. Studies have demonstrated that AI could be
The most common keyword used amongst the top 100 used to reliably detect pulmonary opacities caused by
articles was “deep learning”. DLA are a type of machine COVID-19, as well as differentiating pneumonia caused by
learning, which in itself is composed of an architecture of pathogens other than COVID-19 on thoracic CT.25 Even
ANN and representative learning.3 ANN are based on the before the onset of the COVID-19 pandemic, attempts were
composition of neural networks in the biological brain and made to apply AI to viral pneumonia detection for in-
are generally composed of artificial neurons that can pro- fections, such as H1N1, through the use of support vectors
cess information and distribute communication, similar to for classification and texture analysis of pulmonary opaci-
biological pathways.2 DLA are propelled by data and learn fications on CT.26 Beyond the scope of pandemics, AI has
representative features automatically without human or been used to positive effect in respiratory imaging. Annar-
manual input.4 This is particularly useful for identifying umma et al. demonstrated that convolutional neural net-
human tissue phenotypic features and potential variations works could be used to triage adult chest radiographs for
from expected characteristics.18 When provided with urgent radiologist reporting based on the appearance of
enough data, DLA can detect abnormal or diseased tissue those images, resulting in radiographs with critical findings
H. Hughes et al. / Clinical Radiology 78 (2023) 99e106 105

receiving a radiologist opinion more rapidly than standard studies, as well as potential future trends in this fast-
clinical practice without AI input.27 developing area of radiology.
The USA dominates the top 100 list as the most common
country of publication production. This is consistent with Conflict of Interest
research output in other fields.28,29 This highlights the sig-
nificant role that the USA plays in relation to the output of The authors declare no conflict of interest.
scientific research. Furthermore, it has been reported that
American authors are more likely to cite other American Appendix A. Supplementary data
authors, thereby increasing their prevalence amongst the
most cited papers in the literature.30,31 Contributions were Supplementary data to this article can be found online at
also made by China and Europe to the top 100 list; however, https://doi.org/10.1016/j.crad.2022.09.133.
they were each less than half of that made by the USA.
It is clear that ongoing and improved collaboration be- References
tween the fields of computer technology and radiology is
1. Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine.
imperative in the coming tears in order to ensure collabo-
Gastrointest Endosc 2020;92(4):807e12.
rative intelligence gathering is maintained and to promote 2. Bengio Y, Courville A, Vincent P. Representation learning: a review and new
collective AI research. This may take the form of dual quali- perspectives. IEEE Trans Pattern Anal Mach Intell 2013;35(8):1798e828.
fication in both health sciences and information technology 3. Marblestone AH, Wayne G, Kording KP. Toward an integration of deep
or computer sciences.7 According to Liew et al., keeping learning and neuroscience. Front Comput Neurosci 2016;10:94.
4. Hosny A, Parmar C, Quackenbush J, et al. Artificial intelligence in radi-
software developers in the dark in relation to radiological
ology. Nat Rev Cancer 2018;18(8):500e10.
expertise in order to protect the speciality should not be a 5. Liew C. The future of radiology augmented with artificial intelligence: a
promoted practice.5 Rather, the overarching aim is to find strategy for success. Eur J Radiol 2018;102:152e6.
ways to integrate AI safely into clinical imaging and to 6. Oo AM, ChuT TS. Bibliometric analysis of the top 100 cited articles in head
strengthen the role of the radiologist in providing the vital and neck radiology. Acta Radiol Open 2021;10(3):20584601211001815.
7. Sreedharan S, Mian M, Robertson RA, et al. The top 100 most cited ar-
human component within the clinical context, i.e., through ticles in medical artificial intelligence: a bibliometric analysis. J Med Artif
the creation of “radiologist-in-the-loop systems”.5 Intell 2019;3.
To the authors’ knowledge, this is the first bibliometric 8. Crockett MT, Browne RF, MacMahon PJ, et al. 100 classic papers of inter-
analysis focusing solely on the use of AI in radiology. The aim ventional radiology: a citation analysis. World J Radiol 2015;7(4):79e86.
9. Clarivate. Journal citation Reports. 2020 (accessed 21 November 2021),
of this paper is to provide clinicians and researchers with a list
https://clarivate.com/webofsciencegroup/solutions/journal-citation-
of the most influential publications in this field. It also pro- reports/?gclid¼CjwKCAjwvsqZBhAlEiwAqAHElbAHwcO2Cv
vides an overview and analysis of how trends in this area have RbLAN4PfEfBh0NaY4NSEfoXgVKNL8e9C2wytpsivFtJxoCrmEQAvD_BwE.
evolved over time, as well as advances in AI in radiology. 10. Gonz alez-Pereira B, Guerrero-Bote VP, Moya-Anego  n F. A new approach
This study has a number of limitations. Firstly, some to the metric of journals” scientific prestige: the SJR indicator. J Informetr
2010;4(3):379e91.
pertinent articles may have been excluded or missed due to 11. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical
the search terms used in the search strategy. The search image analysis. Med Image Anal 2017;42:60e88.
strategy used both broad search terms (e.g., artificial intel- 12. Zijdenbos AP, Dawant BM, Margolin RA, et al. Morphometric analysis of
ligence) and specific search terms (e.g., convoluted neural white-matter lesions in MR-imagesdmethod and validation. IEEE Trans
Med Imaging 1994;13(4):716e24.
network) in order to reduce this risk. There is a clear source
13. Menze BH, Jakab A, Bauer S, et al. The multimodal Brain tumor image
of bias in the use of citation count, as the longer a paper has segmentation benchmark (BRATS). IEEE Trans Med Imaging
been published the more likely it is to have been cited, and 2015;34(10):1993e2024.
therefore have a higher citation count. In order to mitigate 14. Kirch DG, Petelle K. Addressing the physician shortage: the peril of
this source of bias, the average citation count per year was ignoring demography. JAMA 2017;317(19):1947e8.
15. Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare
also used. Furthermore, using 2020 journal impact factors
applications. Artificial intelligence in healthcare. 2020. p. 25e60. https://
does not take into consideration potential fluctuations in doi.org/10.1016/B978-0-12-818438-7.00002-2.
impact factor over time. In addition, this study did not ac- 16. Boland GW, Guimaraes AS, Mueller PR. The radiologist’s conundrum:
count for potential bias via self-citation. Journal, author, and benefits and costs of increasing CT capacity and utilization. Eur Radiol
institutional self-citation is recognised as a means of pro- 2009;19(1):9e11. ; discussion 12.
17. McDonald RJ, Schwartz KM, Eckel LJ, et al. The effects of changes in
ducing a boost in citation, thereby positively affecting the utilization and technological advancements of cross-sectional imaging
impact factor. This paper should not be considered be an on radiologist workload. Acad Radiol 2015;22(9):1191e8.
exhaustive list of the top 100 publications in this field; 18. Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu
however, it does serve to highlight some of the most Rev Biomed Eng 2017;19:221e48.
19. Ghafoorian M, Karssemeijer N, Heskes T, et al. Location sensitive deep
influential papers to date.
convolutional neural networks for segmentation of white matter
In conclusion, significant advances have been made in hyperintensities. Sci Rep 2017;7(1):5110.
the field of AI and its incorporation into radiological practice 20. Abajian A, Murali N, Savic LJ, et al. Predicting treatment response to
in order to maximise efficiency and improve patient out- intra-arterial therapies for hepatocellular carcinoma with the use of
comes. This study provides an in-depth analysis of the top supervised machine learningdan artificial intelligence concept. J Vasc
Interv Radiol 2018;29(6):850e7. e851.
100 most-cited papers on the use of AI in radiology. It also 21. Asadi H, Dowling R, Yan B, et al. Machine learning for outcome predic-
provides researchers with detailed insight in to the current tion of acute ischemic stroke post intra-arterial therapy. PLoS One
influential papers in this field, the characteristics of those 2014;9(2):e88225.
106 H. Hughes et al. / Clinical Radiology 78 (2023) 99e106

22. Asadi H, Kok HK, Looby S, et al. Outcomes and complications after 26. Yao J, Dwyer A, Summers RM, et al. Computer-aided diagnosis of pul-
endovascular treatment of brain arteriovenous malformations: a prog- monary infections using texture analysis and support vector machine
nostication attempt using artificial intelligence. World Neurosurg classification. Acad Radiol 2011;18(3):306e14.
2016;96:562e9. e561. 27. Annarumma M, Withey SJ. Automated triaging of adult chest radiographs
23. Iezzi R, Goldberg SN, Merlino B, et al. Artificial intelligence in inter- with deep artificial neural networks. Radiology 2019;291(1):196e202.
ventional radiology: a literature review and future perspectives. J Oncol 28. Kelly JC, Glynn RW, O”Briain DE, et al. The 100 classic papers of orthopaedic
2019;2019:6153041. surgery: a bibliometric analysis. J Bone Jt Surg Br 2010;92(10):1338e43.
24. Else H. How a torrent of COVID science changed research publishingdin 29. Paladugu R, Schein M, Gardezi S, et al. One hundred citation classics in
seven charts. Nature 2020;588(7839):553. https://doi.org/10.1038/ general surgical journals. World J Surg 2002;26(9):1099e105.
d41586-020-03564-y. 30. Paris G, De Leo G, Menozzi P, et al. Region-based citation bias in science.
25. Bai HX, Wang R. Artificial intelligence augmentation of radiologist Nature 1998;396(6708):210.
performance in distinguishing COVID-19 from pneumonia of other 31. Seglen PO. Citation rates and journal impact factors are not suitable for
origin at chest CT. Radiology 2020;296(3):E156e65. evaluation of research. Acta Orthop Scand 1998;69(3):224e9.

You might also like