Artificial Intelligence (AI) for Screening Mammography, From the
AJR Special Series on AI Applications
Leslie R. Lamb, MD, MSc1, Constance D. Lehman, MD, PhD1, Aimilia Gastounioti, PhD2,3, Emily F. Conant, MD2,
Manisha Bahl, MD, MPH1
Breast Imaging · Special Series Review
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
Keywords
artificial intelligence, breast cancer,
implementation, machine learning,
screening mammography
Submitted: Nov 1, 2021
Revision requested: Nov 15, 2021
Revision received: Dec 14, 2021
Accepted: Jan 3, 2022
First published online: Jan 12, 2022
C. D. Lehman receives institutional research
support from the Breast Cancer Research
Foundation, GE Healthcare, and Hologic
and is a cofounder of Clairity. A. Gastounioti
receives research support from iCAD. E. F.
Conant receives research support from
Hologic, iCAD, and OM1; is a member of
advisory panels of Hologic and iCAD; and
has received speaker fees from AuntMinnie.
com. M. Bahl is a consultant for Lunit and
an expert panelist for 2nd.MD. The
remaining author declares that there are no
additional disclosures relevant to the
subject matter of this article.
The content is solely the responsibility of
the authors and does not necessarily
represent the official views of the NIH or
the Susan G. Komen Foundation.
Supported by the Susan G. Komen
Foundation (grant PDF17479714 to A.
Gastounioti) and NIH (grant K08CA241365
to M. Bahl).
Lamb et al.
AI for Screening Mammography
Lamb LR, Lehman CD, Gastounioti A, Conant EF, Bahl M
Breast Imaging
Special Series Review
ARRS is accredited by the Accreditation Council for Continuing
Medical Education (ACCME) to provide continuing medical
education activities for physicians. The ARRS designates this
journal-based CME activity for a maximum of 1.00 AMA PRA
Category 1 Credits™ and 1.00 American Board of Radiology©,
MOC Part II, Self-­Assessment CME (­SA-CME). Physicians should
claim only the credit commensurate with the extent of their
participation in the activity. To access the article for credit, follow
the prompts associated with the online version of this article.
doi.org/10.2214/AJR.21.27071
AJR 2022; 219:369–381
ISSN-L 0361–803X/22/2193–369
© American Roentgen Ray Society
AJR:219, September 2022
Artificial intelligence (AI) applications for screening mammography are being mar-
keted for clinical use in the interpretative domains of lesion detection and diagnosis,
triage, and breast density assessment and in the noninterpretive domains of breast
cancer risk assessment, image quality control, image acquisition, and dose reduction.
Evidence in support of these nascent applications, particularly for lesion detection and
diagnosis, is largely based on multireader studies with cancer-enriched datasets rather
than rigorous clinical evaluation aligned with the application’s specific intended clini-
cal use. This article reviews commercial AI algorithms for screening mammography that
are currently available for clinical practice, their use, and evidence supporting their per-
formance. Clinical implementation considerations, such as workflow integration, gov-
ernance, and ethical issues, are also described. In addition, the future of AI for screening
mammography is discussed, including the development of interpretive and noninter-
pretive AI applications and strategic priorities for research and development.
Artificial intelligence (AI) for breast cancer screening has rapidly advanced from feasi-
bility and reader studies to clinical implementation. As of December 2021, there are more
than 15 FDA-approved AI models for screening mammography [1]. These models can be
used for interpretive applications, including lesion detection and diagnosis, triage, and
density assessment, and for noninterpretive applications, including risk assessment, im-
age quality control, image acquisition, and dose reduction [1–7]. AI, a branch of comput-
er science, includes machine learning (ML), which refers to computers learning from data
without being explicitly programmed, and deep learning (DL), which relies on multilayer
neural networks to extract features from raw data [8]. The most common network for im-
age analysis is the convolutional neural network (CNN). The history of AI, DL, and CNNs can
be traced back several decades; however, recent advances in computational resources,
pivotal developments in AI algorithms, and the growth of digital health data have fueled
a revolution that has brought AI to the mainstream in computational medical imaging [9].
Although multiple trials have proved that screening mammography leads to fewer
breast cancer deaths, digital mammography (DM) remains an imperfect tool [10]. For ex-
ample, the sensitivity of DM is estimated to be 87% [11]. In the United States, 1 in 10 wom-
en is recalled from screening mammography for further evaluation, but only 5% of those
recalled have breast cancer [12]. Furthermore, radiologist performance varies widely, with
41% exceeding recommended recall rates [11]. In addition, the shortage of subspecialized
breast imaging radiologists and the longer interpretation times required for digital breast
tomosynthesis (DBT) have fueled interest in approaches that improve efficiency without
adversely affecting performance [13]. AI algorithms have the potential to improve the in-
terpretation of screening mammography by increasing sensitivity for breast cancer detec-
tion, reducing avoidable recalls and biopsies, and decreasing interpretation times.
Clinical implementation of AI algorithms has progressed rapidly, leading to concerns
that the pace has been ahead of rigorous evaluation of approved algorithms and their
impact on patient outcomes [14–16]. This article reviews commercial AI algorithms for
screening mammography, their use, and evidence supporting their performance. Clinical
1
Department of Radiology, Massachusetts General Hospital, 55 Fruit St, WAC 240, Boston, MA 02114. Address corre-
spondence to M. Bahl (mbahl1@mgh.harvard.edu).
Department of Radiology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA.
2
Present affiliation: Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, MO.
3
www.ajronline.org | 369
 Lamb et al.
implementation considerations are also discussed, in addition to
the future of AI for screening mammography.
Learning From Our Past
Computer-aided detection (CAD) became a major focus for
mammography in the 1980s and 1990s [17]. The first CAD system
gained FDA approval in 1998 and was integrated into practices
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
across the United States after reimbursement for its use was grant-
ed by the Centers for Medicare & Medicaid Services in 2002 [18,
19]. The studies leading to FDA approval and early adoption were
largely reader studies and single-center retrospective studies, but
multiple subsequent studies have shown that CAD in clinical prac-
tice does not improve any metric of screening performance [18,
20]. In particular, CAD has a high false-positive rate, which has led
radiologists to disregard most of its annotations [19].
An early postimplementation study of CAD in the Unit-
ed States by the Breast Cancer Surveillance Consortium (BCSC)
showed that CAD was associated with a decrease in AUC from
0.92 to 0.87 (p = .005) [20]. Specificity was reduced from 90.2%
without CAD to 87.2% with CAD (p < .001), and sensitivity was un-
changed (80.4% vs 84.0%, p = .32) [20]. A subsequent larger BCSC
study with DM reported similar cancer detection rates with CAD
or without CAD [18]. Postimplementation clinical studies did sug-
gest, however, that single reading with CAD could replace double
reading without CAD in certain practices [21, 22].
Fig. 1—Diagram shows comparison of traditional machine learning and deep learning for assessment of mammogram. Deep learning–based algorithm self-learns
imaging features for image analysis task in lieu of relying on human-engineered features. (Image by Loomis S, used with permission)
370
HIGHLIGHTS
„ Commercial artificial intelligence (AI) interpretive
applications for screening mammography have been
shown to have high diagnostic accuracy in
multireader studies.
„ Evidence supporting these applications is largely based
on multireader studies with cancer-enriched datasets;
however, rigorous clinical evaluation aligned with the
application’s intended use is needed.
„ Steps to support successful implementation of an AI tool
into clinical practice include internal validation, workflow
integration, user education, and continuous monitoring.
The primary feature that differentiates new DL-based AI al-
gorithms from traditional CAD is that a DL-based AI algorithm
identifies the imaging features that are useful for a particular im-
age analysis task, whereas the features used by traditional CAD
are human-derived [16, 23] (Fig. 1). DL-based AI algorithms can,
therefore, uncover features and relationships among features
that are not perceived by humans [8]. DL has revived the promise
of AI and has pervaded mammographic screening as one of the
most promising computerized breast imaging tools.
AJR:219, September 2022
 AI for Screening Mammography

TABLE 1: Examples of FDA-Approved Lesion Detection and Diagnosis Applications for Screening
Mammography a
Reported AUCs:
Radiologists Aided
No. of No. of Radiologists Versus Radiologists FDA Type of
Tool (Company) Cases in Reader Study Unaided Class Mammogram Vendor
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved

MammoScreen 2.0 240 for DM 14 for DM 0.80 vs 0.77 for DM II DM and DBT GE Healthcare (DM)
(Therapixel)b [25, 26] 240 for DBT 20 for DBT 0.83 vs 0.79 for DBT Hologic (DM and DBT)
Genius AI Detection 390 17 0.83 vs 0.79 II DBT Hologic
(Hologic) [27]
ProFound AI Software 260 24 0.85 vs 0.80 II DBT GE Healthcare, Hologic,
3.0 (iCAD)c [28, 29] Siemens Healthineers
Transpara 1.7.0 240 for DM 14 for DM 0.89 vs 0.87 for DM II DM and DBT GE Healthcare (DM), Philips Healthcare
(ScreenPoint Medical)d 240 for DBT 18 for DBT 0.86 vs 0.83 for DBT (DM), Fujifilm (DM and DBT),
[30, 33, 100] Hologic (DM and DBT), Siemens
Healthineers (DM and DBT)
Lunit INSIGHT MMG 240 12 0.81 vs 0.75 II DM GE Healthcare, Hologic, Siemens
(Lunit) [31] Healthineers
Note—DM = digital 2D mammography, DBT = digital breast tomosynthesis.
a
The information in this table is based on the referenced FDA documentation and, where indicated, verification by company representatives.
b
De Snoeck Q, Therapixel representative, written communication, 2021.
c
Hawkins R, iCAD representative, written and oral communications, 2021.
d
Karssemeijer N, ScreenPoint Medical representative, written communication, 2021.

Commercial Artificial Intelligence Algorithms
Approved for Clinical Practice
Lesion Detection and Diagnosis
AI-based CAD systems offer both computer-aided detection
(which refers to identification of lesions) and computer-aided di-
agnosis (which refers to classification of lesions as likely benign
or malignant) [24]. As of December 2021, there are at least five
FDA-approved AI applications for lesion detection and diagno-
sis [1, 25–31] (Table 1). These tools offer traditional lesion mark-
ers with corresponding level-of-suspicion scores, in addition to
scores at the breast and/or examination levels (Fig. 2). Four of the
five tools can be used for DBT examinations.
Evidence supporting these applications is largely based on
reader studies with cancer-enriched datasets. Although early re-
search is promising, postimplementation evaluation in clinical
settings is limited. Factors that may affect the utility of AI applica-

Fig. 2—Example of lesion detection and diagnosis

by artificial intelligence (AI) algorithm for digital
breast tomosynthesis in 76-year-old woman.
A and B, Right craniocaudal (A) and mediolateral
oblique (B) views show focal asymmetry (box) in
lower, slightly inner quadrant of right breast, which
was detected by AI algorithm and assigned lesion
score (42, A; 64, B) reflecting percentage likelihood
of malignancy (i.e., 42% on craniocaudal view and
64% on mediolateral oblique view). Histopathology
showed grade 2 invasive ductal cancer. Asymmetry
in lateral right breast was also detected by AI
algorithm (circle, A) and assigned lesion score of
8 (A; i.e., 8% likelihood of malignancy); this latter
finding was deemed benign on basis of long-term
mammographic stability.
A B

AJR:219, September 2022 371

 Lamb et al.

1.0
Sensitivity or True-Positive Rate

0.8
20

Reading Time Difference (s)

5
0.6 0. 0
of
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved

C
AU –20
ith
0.4 erw
s ifi –40
as
cl
om
nd –60
0.2 Ra Unaided (0.866)
AI support (0.886) –80

0
–100
0 0.2 0.4 0.6 0.8 1.0 Readers
1 – Specificity or False-Positive Rate

Fig. 3—Mean ROC curves with artificial intelligence (AI) support from lesion
detection and diagnosis algorithm and without AI support. Means are calculated
across 14 radiologists who participated in reader study [33]. Values in parentheses
are AUCs. Adapted from Rodríguez-Ruiz A, Krupinski E, Mordang JJ, Schilling
K, Heywang-Köbrunner SH, Sechopoulos I, Mann RM, 2019, doi.org/10.1148/
radiol.2018181371, according to CC BY 4.0 license (creativecommons.org/licenses/
by/4.0/legalcode), including disclaimer in Section 5, with adjustments to wording
of axis labels, adjusted coloring within graph, addition of text within graph,
removal of grid lines, and horizontal widening of graph area.
Fig. 4—Bar chart shows differences between mammographic reading times
with artificial intelligence (AI) support and without AI support in reader study
of 24 radiologists. Reading time improved with AI support, with mean decrease
of 35 seconds. Each blue bar represents individual reader. Only one reader
(leftmost bar) had increase in reading time with AI support (1.4 seconds). Data
shown were extracted from Conant et al. [35]. (Image by Loomis S, used with
permission)

tions in practice include the radiologist’s confidence in the AI sys- er, these results come from a reader study without simulation of
tem and in their own interpretation, the radiologist’s training and routine clinical workflow [35] (Fig. 4).
experience in interpreting mammography with and without AI A head-to-head comparison of three AI-based CAD systems
support, interaction with the AI system, and the transparency of reported AUCs of 0.96, 0.92, and 0.92, respectively, with sensitiv-
the rationale being used by the AI system for its predictions [32]. ities of 81.9%, 67.0%, and 67.4% when matching the specificity of
In a multireader study evaluating one of the commercial appli- the radiologists [36]. Best performance was achieved, however, by
cations, the AUC was reported to be higher with the use of the AI combining the high-performing AI system with the expertise of ra-
system (0.89 vs 0.87, p = .002), although this increase was not as diologists. In a highly publicized report from Google Health and
pronounced for experienced radiologists [33] (Fig. 3). Sensitivity DeepMind Technologies, their AI system (not currently approved
improved with AI support compared with sensitivity without AI by the FDA) outperformed all six radiologists in a reader study, with
support (86% vs 83%, p = .046), and there was a nonsignificant in- the AI system’s AUC shown to be higher than the mean radiolo-
crease in specificity (79% vs 77%, p = .06) [33]. A multireader study gist AUC by an absolute margin of 11.5% [37]. The limitations of
of the same AI system found that its stand-alone performance that study include the use of a nonrepresentative U.S. dataset (can-
was not inferior to that of 101 radiologists (AUC, 0.84 vs 0.81) [34]. cer-enriched and from a single site) and images from a single ven-
With regard to AI-based CAD for DBT, a multireader study of a dor. A systematic review of published reports on lesion detection
commercial AI system (that provides localization of suspicious and diagnosis algorithms, such as the aforementioned algorithm,
findings within the DBT dataset) reported a reduction in mean found that most studies in this domain have poor methodologic
reading time with AI support (64.1 vs 30.4 seconds) [35]; howev- quality with applicability concerns and high bias risk [15].

TABLE 2: Examples of FDA-Approved Triage Applications for Screening Mammography a

No. of Cases in
Tool (Company) Performance Study Reported AUCs FDA Class Type of Mammogram Vendor
Saige-Q (DeepHealth) [38] b
1333 for DM and 0.97 for DM and II DM and DBT Hologic
1528 for DBT 0.99 for DBT
cmTriage (CureMetrix) [39] 1255 0.95 II DM Agnostic
HealthMammo (Zebra Medical Vision) [40] 835 0.97 II DM Hologic
Note—DM = digital 2D mammography, DBT = digital breast tomosynthesis.
a
The information in this table is based on the referenced FDA documentation and, in one case, verification by a company representative.
b
Lotter W, DeepHealth representative, written communication, 2021.

372 AJR:219, September 2022

 AI for Screening Mammography

TABLE 3: Examples of FDA-Approved Density Assessment Applications for Screening Mammography a

Tool (Company) FDA Class Type of Mammogram Vendor
DM-Density (Densitas) [45]
b
II DM GE Healthcare, Hologic, Siemens Healthineers
densityai (Densitas)b [46] II DM and synthetic DM GE Healthcare (DM), Siemens Healthineers (DM), Hologic (DMc)
DenSeeMammo (Statlife) [47] II DM GE Healthcare, Hologic
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved

Insight BD (Siemens Healthineers) [48] II DM and DBT Siemens Healthineers

PowerLook Density Assessment 2.1 and 4.0 II DM (2.1) and synthetic DM (4.0) Siemens Healthineers (DM), GE Healthcare (DMe), Hologic (DMc)
(iCAD)d [49]
Quantra 2.1 and 2.2 (Hologic) [50] II DM (2.1 and 2.2) and DBT (2.2) GE Healthcare (2.1), Siemens Healthineers (2.1), Hologic (2.1 and
2.2)
Visage Breast Density (Visage Imaging) [51] II DM and DBT Hologic
Volpara Imaging Software (Volpara Health II DM and DBT Canon (DM), Fujifilm (DM), Medi-Future (DM), Metaltronica
Technologies)f [52] (DM), Philips Healthcare (DM), PlanMed (DM), Sectra (DM), GE
Healthcare (DM and DBT), Hologic (DM and DBT), IMS Giotto
(DM and DBT), Siemens Healthineers (DM and DBT)
WRDensity (Whiterabbit.ai) [53] II DM and synthetic DM Hologic
Note—DM = digital 2D mammography, DBT = digital breast tomosynthesis.
a
The information in this table is based on the referenced FDA documentation and, where indicated, verification by company representatives.
b
Abdolell M, Densitas representative, written communication, 2021.
c
With C-View software (Hologic).
d
Hawkins R, iCAD representative, written and oral communications, 2021.
e
With V-Preview software (GE Healthcare).
f
Chan A, Volpara Health Technologies representative, written communication, 2021.

Triage served reduction. The authors also highlighted “a perception of

Triage tools can be used to prioritize patients within the PACS
worklist by flagging mammograms with one or more suspicious
findings. At least one of the previously described applications can
also be used for triage purposes (specifically, the case score can
be used for prioritization) [29]. In addition, at least three AI ap-
plications that are specifically marketed for triage have been ap-
proved by the FDA as of December 2021 [1, 38–40] (Table 2). One
of the triage tools can be used for both DBT and DM examina-
tions [38]. Two of the tools can be used only with Hologic, and
one tool is vendor-agnostic.
For one of the triage tools, performance data cited in the FDA
Summary are based on a multicenter cancer-enriched dataset
[39]. The AUC is 0.95 and is similar across different breast densities
and mammographic lesion types [39]. In a published case study
based on 2 years of clinical experience with this tool, improve-
ments in mean turnaround time from examination completion
to reporting (from 9.6 days in 2019 to 3.9 days in 2021, p < .05)
were largely attributed to the triage tool [41]; however, other fac-
tors—such as the number of radiologists, their experience, ex-
amination volumes, and workflow changes—could have differed
between 2019 and 2021 and could have contributed to the ob-
greater ease” when interpreting batched mammograms without
any suspicious findings flagged by the tool [41].
Breast Density Assessment
Breast density notification legislation has heightened inter-
est in the importance of dense breast tissue, which is a risk factor
for breast cancer and can also mask cancers on mammography;
however, considerable intrareader and interreader variability ex-
ists with regard to breast density assessment [42–44]. At least
nine AI applications for breast density assessment have been ap-
proved by the FDA as of December 2021 [1, 45–53] (Table 3). The
tools provide numeric density values for each breast, in addition
to providing BI-RADS density categories [54]. The tools may be
used for DM, synthetic DM, or DBT and may be vendor-specific or
vendor-agnostic.
In a head-to-head comparison of three fully automated meth-
ods and visual assessment of breast density, two automated meth-
ods were strongly associated with breast cancer risk, although the
strongest predictor was visual assessment [55]. In a second head-
to-head comparison of two automated methods and visual assess-
ment, all had similar associations with breast cancer risk; however,

TABLE 4: Potential Use Case Scenarios for Improving Breast Imaging Services With Artificial Intelligence
Use Case Purpose
Breast cancer risk assessment Develop a reliable standard for breast cancer risk based on information from multiple sources
Classification of calcifications Detect the morphology and distribution of calcifications on mammography and recommend follow-up based on risk level
Classification of high-risk Automate high-risk lesion classification based on level of suspicion of upgrade to malignancy
breast lesions
Note—The information in this table is based on the Breast Imaging section of the American College of Radiology Data Science Institute website [88].

AJR:219, September 2022 373

 Lamb et al.
1.0 Perfect classifier with AUC of 1.0
Sensitivity or True-Positive Rate
0.8 Optimal cutoff point
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
0.6
High specificity
0.4
om
nd
0.2 Ra
0
0 0.2
1 – Specificity or False-Positive Rate
Fig. 5—Example of ROC curve. Blue line represents model that is perfect
classifier (AUC = 1.0), and red line represents model that is random classifier
(AUC = 0.5). Most ROC curves fall between these two lines, as shown by green
and purple lines. For any given clinical scenario, sensitivity and specificity are
considered to be optimal at operating point or cutoff point on ROC curve.
(Image by Loomis S, used with permission)
the level of agreement of the two automated methods with visu-
al assessment was only moderate (κ = 0.57 and 0.46, respectively),
with observed differences of 6–14% in dense tissue categorization
[56]. Given these findings, the authors suggested using the same
technique to evaluate individual changes over time, especially
when density serves as a marker for treatment response.
Breast Cancer Risk Assessment
Individualized risk assessment may be used to determine
which patients are at high breast cancer risk and who thus may
benefit from MRI screening, chemoprevention, or genetic test-
ing. In fact, before a patient undergoes screening with MRI, the
patient’s risk level (≥ 20%) must be documented to ensure insur-
ance coverage of the examination [57]. Traditional risk assess-
ment models incorporating patient-specific features are calibrat-
ed at the population level rather than the individual level and
were largely developed based on White women [58]. More re-
cently, mammographic density has been added to risk prediction
models [59].
One of the companies with a commercial AI-based breast den-
sity algorithm has also developed a risk assessment application,
which combines AI-based breast density, breast volume, age,
and other clinical risk factors to generate a current risk prediction
[2]. The AUC for the model based on imaging features (AI-based
breast density and breast volume) has been reported to be 0.60
[60]. Similarly, another company has combined AI-based breast
density (which is FDA-approved) with a traditional risk prediction
model (Tyrer-Cuzick, version 8, which does not require FDA ap-
proval) to calculate lifetime breast cancer risk [3].
Breast density alone, however, does not capture the rich infor-
mation contained in a mammogram [61]. At least one commer-
cial risk prediction model uses age and more information from
the mammogram than just density, including mammograph-
374
High sensitivity
5
0.
of
AUC
with
fi er
si
as
cl
0.4 0.6 0.8 1.0
Fig. 6—Example of confusion matrix. Confusion matrix allows reader to easily
determine how artificial intelligence algorithm is performing. In this example,
algorithm predicts true class of 2 with 83% accuracy. (Image by Loomis S, used
with permission)
ic features such as masses and calcifications and differences in
the right and left breasts [4]. This 2-year risk prediction model is
marketed as a clinical decision support tool, thereby exempting
it from FDA regulation [62]. The DM-based model has an AUC of
0.73 and was externally validated in three patient cohorts [63]. A
1-year risk prediction model based on DBT images by the same
company was recently released [4]. These short-term risk predic-
tion models could potentially be used to determine screening in-
tervals (1 year vs 2 years) for individual women.
Image Quality Control, Image Acquisition, and Dose
Reduction
The application of AI to the image acquisition process can im-
prove image quality, process efficiency, and technologist perfor-
mance [5, 6, 19]. For example, a commercial AI application (not
requiring FDA approval) provides real-time patient position and
compression feedback to technologists [6]. Positioning parame-
ters include nipple in profile, visibility of the inframammary fold, vi-
sualization of the pectoralis muscle extending inferiorly to at least
1 cm above the posterior nipple line, and adequate visualization
of the pectoralis muscle angle, length, and width. With real-time
feedback, repeat imaging can be immediately performed, thus re-
ducing technical recalls. This AI system also provides automated
analytic reports with image quality performance metrics. In addi-
tion, DL-based reconstruction of synthetic images from DBT exam-
inations is currently being performed and has led to radiation dose
reductions (since 2D images need not be obtained) [7].
Clinical Implementation
Clinical Adoption
The potential of AI for screening mammography has not yet
been realized, with most research based on reader studies with
cancer-enriched datasets and limited postimplementation clin-
AJR:219, September 2022
 AI for Screening Mammography
ical evaluation [15, 16, 33–35, 37]. Rigorous clinical evaluation
aligned with the AI tool’s specific intended use will help deter-
mine its true impact on patient outcomes. In addition to the lack
of evidence and therefore uncertainty about the value of AI in
clinical practice, other obstacles to adoption include concerns
about decreased clinical productivity with AI, inconsistent tech-
nical performance of AI systems, the absence of guidelines and
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
best systems, the variability in the level of trust in AI by radiol-
ogists, other clinicians, and patients [64, 65]. Practical factors to
consider include the mammographic views and image format
required for model processing, compatibility of the model with
local practice techniques (e.g., DM, synthetic DM, or DBT) and
mammography equipment, and algorithm processing time and
storage needs in the context of local resources. When purchas-
ing an AI tool, cost considerations include software and hardware
versus only software acquisitions, pricing based on the number
of mammography machines and/or the volume of examinations,
and the possibility of additional charges associated with software
updates and license transfers.
Adoption of AI is likely to be influenced by radiologists’ knowl-
edge, perception, and attitude. For example, a survey of radiol-
ogists and radiology residents found that limited AI knowledge
was associated with fear, whereas greater AI knowledge was re-
lated to a positive attitude toward AI, suggesting that exposure
and training may help accelerate clinical adoption [66]. Accord-
ing to a 2020 survey of 1427 American College of Radiology (ACR)
members, one-third of radiologists were using AI as part of their
practice, and most respondents who were not using AI respond-
ed that they “see no benefit” to AI [65]. Survey respondents ex-
pressed a desire for methods to evaluate and report the per-
formance of AI tools on representative datasets, evaluate tools
before purchase, and monitor their accuracy over time. Address-
ing radiologists’ concerns and mitigating barriers to adoption will
provide opportunities for the future growth of AI applications.
Evaluation
Implementation of AI tools requires that the tool works (per-
formance validation), that the user understands the tool (trust),
and that the tool is integrated into the existing workflow (user
experience design) [19]. The performance of an AI tool can be as-
sessed using ROC curves (Fig. 5) and/or confusion matrices [8]
(Fig. 6). After installation of an AI tool, a practice can perform its
own validation by allowing a test period during which radiolo-
gists do not have access to AI output [67]. Radiologists can later
review the predictions made by the AI tool and evaluate its per-
formance across a large volume of cases. This test period will also
facilitate learning of the strengths and limitations of the tool and
allow users to gain trust in the AI tool’s predictions. Even the most
trusted AI tool will fail, however, if not appropriately integrated
into existing workflows [19].
Workflow Integration
Steps involved in integrating an AI tool into clinical practice in-
clude internal validation, workflow integration, user education,
and continuous monitoring [68]. The first step involves the practice
conducting its own internal validation of the AI tool, since regula-
tory approval does not ensure that the tool will be generalizable
to all practices [67, 69]. The integration phase involves assimilating
AJR:219, September 2022
the AI tool into the PACS and reporting systems workflow in col-
laboration with the AI vendor and the practice’s informatics team
[68]. To increase awareness and acceptance, education should in-
volve not only radiologists but also technologists and administra-
tors. The last phase is continuous monitoring of the AI tool after de-
ployment to ensure adequate performance [68, 69].
Certain AI tools for screening mammography may be integrat-
ed into the clinical workflow based on local practices and needs
(Fig. 7). For example, a lesion detection and diagnosis application
could replace the second reader at sites that offer double-read-
ing or could potentially be used for stand-alone interpretation
in communities with limited breast imaging expertise, though
none of the commercial tools are currently approved for stand-
alone interpretation [25–31]. Practices could also use AI tools
to prioritize examinations to be read immediately or to gener-
ate customized worklists for different readers. One of the com-
mercial tools for lesion detection and diagnosis, for example, of-
fers a case complexity index (multiple findings, single finding, or
no findings), a read time indicator (high, medium, or low), and a
reading priority indication (which flags the most concerning cas-
es), which are metrics that can be used to categorize cases and
enhance workflow [70]. Algorithms that provide imaging data–
driven risk scores (incorporating lesion scores, asymmetries be-
tween breasts, and quantitative density measures) could be used
to help refine recommendations for supplemental screening [59].
Multidisciplinary Collaboration
Essential to the development and implementation of AI algo-
rithms is a strong relationship between domain experts (i.e., ra-
diologists) and technical experts (i.e., data scientists, engineers)
who may be in the academic or private sectors [71, 72]. Radiolog-
ic technologists have also expressed a strong interest in being in-
volved in development and implementation [73]. Communication
among the multidisciplinary team members is necessary to under-
stand and agree on the goals of the algorithm. The potential of AI
will be realized only when these algorithms are deployed in prac-
tice; adopted by radiologists, other clinicians, technologists, and
patients; and their impact is measured using relevant and specific
metrics. Radiologists must ultimately determine the intended im-
pact of and clinically relevant metrics for the AI system and should
continue to communicate with the model developers [71].
Governance
The FDA regulates AI algorithms that are medical devices and
defines “medical devices,” according to the 21st Century Cures Act,
as follows: “[tools] intended for use in the diagnosis of disease or
other conditions, or in the cure, mitigation, treatment, or preven-
tion of disease… or intended to affect the structure or any function
of the body of man or other animals…” [74, 75]. AI tools used in the
interpretation of imaging, such as the applications for lesion detec-
tion and diagnosis, triage, and density assessment, are thus regu-
lated by the FDA. The FDA classifies these devices into one of three
classes according to their risk level and intended use. Approved AI
algorithms for mammography are categorized as class II devices,
which are considered to be moderate risk [1, 76].
Certain AI algorithms such as those considered to be clinical
decision support software are exempt from FDA regulation [62].
Clinical decision support software is a tool that provides the fol-
375
 Lamb et al.
lowing: “knowledge and person-specific information, intelligent-
ly filtered or presented at appropriate times to enhance health
and health care” [62]. The software can offer recommendations to
a health care professional, but it cannot be used to replace pro-
fessional judgment. The previously discussed short-term risk pre-
diction tool and the image quality control tools are considered
clinical decision support software and are therefore not subject
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
to FDA regulation [4–6].
Ethical and Medicolegal Issues
The use of AI in health care raises ethical questions with regard
to algorithm bias, resource allocations, conflicts of interest, and
the doctor-patient relationship, in addition to liability concerns.
Recent research has shown that AI algorithms have been dispro-
portionately trained on patients from New York, Massachusetts,
and California, raising concern that these algorithms may gener-
alize poorly and thus highlighting the importance of validation
across diverse populations [77]. Regarding resource allocations,
the development and implementation of AI tools require large
datasets, the technology and technical skills to manage the data,
and the computational power to develop and maintain AI sys-
Fig. 7—Diagram shows integration of artificial intelligence (AI) into screening workflow. AI applications can be used for noninterpretive functions, such as breast
cancer risk assessment, image quality control, image acquisition, and dose reduction, and for interpretive purposes, such as triage, breast density assessment, and
lesion detection and diagnosis. (Image by Loomis S, used with permission)
376
tems. Thus, small and/or resource-poor practices may not be able
to take advantage of AI systems, which could result in disparities
in the use of AI and its potential benefits [78]. As with all agree-
ments between commercial entities and health care organiza-
tions, potential conflicts of interests between the institution’s
members and the company must be monitored following stan-
dard practices of public disclosure, institutional monitoring, and
recusal of members with a significant conflict of interest from the
decision-making process [79].
The integration of AI into clinical practice will undoubtedly im-
pact the doctor-patient relationship, and uncertainty exists re-
garding patients’ acceptance of AI. In one survey, more than three-
fourths of women did not support the use of a fully autonomous
AI system for screening mammography [80]. Survey respondents
were receptive to the idea of the AI tool serving as a second read-
er, although more than 40% did not believe that AI scores should
be used as a method to determine which patients require a sec-
ond read. Education about the strengths and limitations of AI ap-
plications could help mitigate patient concerns and increase their
acceptance. In terms of practice and radiologist liability, questions
have arisen about the legal responsibility of AI manufacturers for
AJR:219, September 2022
 AI for Screening Mammography
errors made by AI tools. As of December 2021, however, radiolo-
gists continue to bear ultimate responsibility, though the issue of
liability is complex and will continue to evolve [78].
Future of Artificial Intelligence for Screening
Mammography
Artificial Intelligence Model Development
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
Mammographic images present multiple technical challenges
for AI model development. Most efforts have focused on the ap-
plication of existing DL models to mammographic images rather
than the development of new architectures suited to this domain
[81]. However, customized architectures that can leverage intrinsic
characteristics of mammographic imaging, such as the high image
resolution, the multiple mammographic views, and varying recon-
struction algorithms, could further boost model performance [37,
82, 83]. Moreover, extensive work is needed to determine if and
how the models developed for DM can be applied to DBT [81].
Access to high-quality expert annotations (or labels) is also a
critical factor for model development. AI models that learn in a
supervised fashion from both pixel-level annotations (e.g., circle
around cancer) and examination-level annotations (e.g., mam-
mogram labeled as positive or negative for cancer) typically
show better performance [37, 84]. However, obtaining high-qual-
ity pixel-level labels is a major challenge as these annotations can
be time-consuming and tedious, especially with DBT. Recogniz-
ing the need for annotated datasets, the ACR provides resources
in the ACR AI-LAB to annotate datasets [85].
A challenge to be addressed before AI becomes commonplace
in mammographic screening is interpretability [86]. Models that
use an explainable transparent rationale to make predictions are
more likely to be trusted, which can accelerate their adoption
into clinical practice. In addition, AI models must generalize well
to unseen heterogeneous datasets. Therefore, large retrospec-
tive studies involving diverse populations, different vendors, and
various image acquisition settings are essential. Although evalu-
ation with retrospective datasets provides insight into the poten-
tial performance of AI models depending on the intended clinical
use, validation studies in real-world clinical settings are necessary
to fully appreciate the performance of AI applications, their im-
pact on radiologist performance, and the complex interaction
between the two [87].
Future Artificial Intelligence Applications for Screening
Mammography
The ACR Data Science Institute has outlined a series of use case
scenarios for improving breast imaging services with AI [88] (Ta-
ble 4). For example, research is in progress for interpretive AI ap-
plications that detect and classify calcifications on mammogra-
phy, with preliminary results showing that CNNs can outperform
handcrafted features for this task [89]. In addition, an AI algorithm
is being investigated to detect arterial calcifications on mammog-
raphy, which can help identify women at risk of coronary artery
disease [90]. Research is also ongoing for noninterpretive appli-
cations, including long-term risk prediction models (e.g., 5-year,
10-year, and lifetime) that incorporate more mammographic in-
formation than just breast density [91, 92].
Early claims that AI-based stand-alone interpretation of screen-
ing mammograms was imminent proved false, but the goal of AI
AJR:219, September 2022
assisting radiologists with triaging normal cases out of their work
queue remains. In fact, DL models have been shown to confi-
dently classify mammograms as cancer-free, which could lead to
a reduction in radiologist workload [93, 94]. AI applications that
provide rapid and accurate assessment of a mammogram as can-
cer-free may also be valuable in communities that lack access to
breast imaging expertise.
Strategic Priorities for Research and Development
To facilitate AI algorithm development, the following priori-
ties in AI research were identified at an international workshop
organized by the NIH: new image reconstruction and enhance-
ment methods, automated labeling and annotation methods,
novel algorithms developed for the complexity of imaging data,
methods that explain AI advice given to users, and methods for
deidentification of imaging data and data sharing [95]. Priorities
for promoting AI include creating standards and methods for
data sharing and management, improving the user interface and
experience, ensuring patient safety and health equity, and devel-
oping efficient pathways to commercialize AI tools [96]. Active
collaboration among radiologists, model developers, research
scientists, and regulatory bodies is critical to translate research
into practice and to promote AI use.
Value Proposition of Artificial Intelligence for Screening
Mammography
The so-called “volume to value” transition is a primary compo-
nent of the ACR’s 3.0 initiative [97]. AI must show its added val-
ue to mammography through the standard metrics of improved
performance, increased efficiency, and a preferential cost-ben-
efit ratio [19]. To show improved performance, audit reports for
screening mammography must show evidence of increased can-
cer detection rates and decreased recall rates after AI implemen-
tation. Early studies show that one of the benefits of AI-based
CAD over traditional CAD may be higher specificity with fewer
annotations made by the AI system [35, 41]. To increase efficien-
cy, AI algorithms could aid radiologists by detecting and charac-
terizing lesions, autocategorizing breast density, prepopulating
reports, and reducing screening workload by autoreporting nor-
mal examinations [19].
As CAD is currently part of a bundled rather than separate
charge for screening mammograms, the possibility of introduc-
ing a new billing code for AI remains uncertain. Payment in our
fee-for-service environment could pose challenges, but future
value-based payment models may recognize the value of AI tools
that improve quality and equity [98]. In the more immediate fu-
ture, AI could lead to reduced costs through increased efficien-
cy (for example, if a subset of screening mammograms are accu-
rately autoreported as normal without radiologist involvement
or if the perceived need for double-reading by radiologists is
reduced) and/or through a reduction in false-positive examina-
tions, which are estimated to cost nearly $3 billion per year [99].
Conclusion
Breast imaging has been at the forefront of AI in health care,
with AI applications for breast cancer screening having quickly
advanced from feasibility and reader studies to clinical imple-
mentation. Commercial AI applications for screening mammog-
377
 Lamb et al.
raphy are available in the interpretive domains of lesion detec-
tion and diagnosis, triage, and density assessment and in the
noninterpretive domains of risk assessment, image quality con-
trol, image acquisition, and dose reduction. The rapid deploy-
ment of AI tools into clinical practice, however, may outpace the
rigorous evaluation of these tools and their potential impact on
patient care and outcomes. The success of AI for breast cancer
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
screening will require that AI algorithms not only are validated
in the clinical environment but also are appropriately integrat-
ed into existing workflows and show added value to screening
mammography through improved patient outcomes, increased
efficiency, and a preferential cost-benefit ratio.
Acknowledgment
We thank Susanne L. Loomis (Medical and Scientific Commu-
nications, Strategic Communications, Department of Radiology,
Massachusetts General Hospital, Boston, MA) for creating Figures
1 and 4–7 in this article.
References
1. American College of Radiology Data Science Institute website. AI Central.
models.acrdsi.org/. Accessed August 25, 2021
2. Densitas website. Densitas riskaiTM. densitas.health/riskai. Accessed Sep-
tember 15, 2021
3. Volpara Health website. ScorecardTM. www.volparahealth.com/breast-
health-platform/products/scorecard/. Accessed September 15, 2021
4. iCAD website. ProFound AI Risk. www.icadmed.com/profoundai-risk.
html. Accessed September 15, 2021
5. Densitas website. Densitas qualityaiTM. densitas.health/solutions/quality/.
Accessed September 20, 2021
6. Volpara Health website. LiveTM. www.volparahealth.com/breast-health-
platform/products/live/. Accessed September 20, 2021
7. Hologic website. 3DQuorumTM imaging technology. www.hologic.com/
sites/default/files/downloads/WP-00152_Rev001_3DQuorum_Imaging_
Technology_Whitepaper%20%20(1).pdf. Published October 2019. Ac-
cessed October 15, 2021
8. Bahl M. Artificial intelligence: a primer for breast imaging radiologists. J
Breast Imaging 2020; 2:304–314
9. Trister AD, Buist DSM, Lee CI. Will machine learning tip the balance in
breast cancer screening? JAMA Oncol 2017; 3:1463–1464
10. Nyström L, Andersson I, Bjurstam N, Frisell J, Nordenskjöld B, Rutqvist LE.
Long-term effects of mammography screening: updated overview of the
Swedish randomised trials. Lancet 2002; 359:909–919
11. Lehman CD, Arao RF, Sprague BL, et al. National performance benchmarks
for modern screening digital mammography: update from the Breast
Cancer Surveillance Consortium. Radiology 2017; 283:49–58
12. Lee CS, Parise C, Burleson J, Seidenwurm D. Assessing the recall rate for
screening mammography: comparing the Medicare Hospital Compare
dataset with the National Mammography Database. AJR 2018; 211:127–132
13. Dang PA, Freer PE, Humphrey KL, Halpern EF, Rafferty EA. Addition of to-
mosynthesis to conventional digital mammography: effect on image in-
terpretation time of screening examinations. Radiology 2014; 270:49–56
14. Lehman CD. Artificial intelligence to support independent assessment of
screening mammograms: the time has come. JAMA Oncol 2020; 6:1588–
1589
15. Freeman K, Geppert J, Stinton C, et al. Use of artificial intelligence for im-
age analysis in breast cancer screening programmes: systematic review of
test accuracy. BMJ 2021; 374:n1872
378
16. Hickman SE, Woitek R, Le EPV, et al. Machine learning for workflow appli-
cations in screening mammography: systematic review and meta-analy-
sis. Radiology 2022; 302:88–104
17. Giger ML, Chan HP, Boone J. Anniversary paper: history and status of CAD
and quantitative image analysis: the role of Medical Physics and AAPM.
Med Phys 2008; 35:5799–5820
18. Lehman CD, Wellman RD, Buist DS, Kerlikowske K, Tosteson AN, Miglioret-
ti DL; Breast Cancer Surveillance Consortium. Diagnostic accuracy of digi-
tal screening mammography with and without computer-aided detec-
tion. JAMA Intern Med 2015; 175:1828–1837
19. Morgan MB, Mates JL. Applications of artificial intelligence in breast imag-
ing. Radiol Clin North Am 2021; 59:139–148
20. Fenton JJ, Taplin SH, Carney PA, et al. Influence of computer-aided detec-
tion on performance of screening mammography. N Engl J Med 2007;
356:1399–1409
21. Gilbert FJ, Astley SM, Gillan MG, et al.; CADET II Group. Single reading with
computer-aided detection for screening mammography. N Engl J Med
2008; 359:1675–1684
22. Gromet M. Comparison of computer-aided detection to double reading
of screening mammograms: review of 231,221 mammograms. AJR 2008;
190:854–859
23. Bahl M. Detecting breast cancers with mammography: will AI succeed
where traditional CAD failed? Radiology 2019; 290:315–316
24. Gao Y, Geras KJ, Lewin AA, Moy L. New frontiers: an update on comput-
er-aided diagnosis for breast imaging in the age of artificial intelligence.
AJR 2019; 212:300–307
25. FDA website. Approval document for MammoScreen. www.accessdata.
fda.gov/cdrh_docs/pdf19/K192854.pdf. Published March 25, 2020. Ac-
cessed September 10, 2021
26. FDA website. Approval document for MammoScreen 2.0. www.­
accessdata.fda.gov/cdrh_docs/pdf21/K211541.pdf. Published November
26, 2021. Accessed December 29, 2021
27. FDA website. Approval document for Genius AI Detection. www.­
accessdata.fda.gov/cdrh_docs/pdf20/K201019.pdf. Published November
18, 2020. Accessed September 10, 2021
28. FDA website. Approval document for ProFound AI Software V2.1. www.
accessdata.fda.gov/cdrh_docs/pdf19/K191994.pdf. Published October 4,
2019. Accessed September 10, 2021
29. FDA website. 510(k) Premarket Notification for ProFound AI Software
V3.0. www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm?ID=​
K203822. Published March 12, 2021. Accessed October 15, 2021
30. FDA website. Approval document for Transpara 1.7.0. www.accessdata.
fda.gov/cdrh_docs/pdf21/K210404.pdf. Published July 30, 2021. Accessed
September 10, 2021
31. FDA website. Approval document for Lunit INSIGHT MMG. www.­
accessdata.fda.gov/cdrh_docs/pdf21/K211678.pdf. Published November
17, 2021. Accessed December 12, 2021
32. Hsu W, Hoyt AC. Using time as a measure of impact for AI systems: impli-
cations in breast screening. Radiol Artif Intell 2019; 1:e190107
33. Rodríguez-Ruiz A, Krupinski E, Mordang JJ, et al. Detection of breast can-
cer with mammography: effect of an artificial intelligence support sys-
tem. Radiology 2019; 290:305–314
34. Rodríguez-Ruiz A, Lång K, Gubern-Merida A, et al. Stand-alone artificial
intelligence for breast cancer detection in mammography: comparison
with 101 radiologists. J Natl Cancer Inst 2019; 111:916–922
35. Conant EF, Toledano AY, Periaswamy S, et al. Improving accuracy and effi-
ciency with concurrent use of artificial intelligence for digital breast to-
mosynthesis. Radiol Artif Intell 2019; 1:e180096
AJR:219, September 2022
 AI for Screening Mammography
36. Salim M, Wåhlin E, Dembrower K, et al. External evaluation of 3 commer-
cial artificial intelligence algorithms for independent assessment of
screening mammograms. JAMA Oncol 2020; 6:1581–1588
37. McKinney SM, Sieniek M, Godbole V, et al. International evaluation of an AI
system for breast cancer screening. Nature 2020; 577:89–94
38. FDA website. Approval document for Saige-Q. www.accessdata.fda.gov/
cdrh_docs/pdf20/K203517.pdf. Published April 16, 2021. Accessed Sep-
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
tember 10, 2021
39. FDA website. Approval document for cmTriage. www.accessdata.fda.gov/
cdrh_docs/pdf18/K183285.pdf. Published March 8, 2019. Accessed Sep-
tember 10, 2021
40. FDA website. Approval document for HealthMammo. www.accessdata.
fda.gov/cdrh_docs/pdf20/K200905.pdf. Published July 16, 2020. Ac-
cessed September 10, 2021
41. Tartar M, Le L, Watanabe AT, Enomoto AJ. Artificial intelligence support
for mammography: in-practice clinical experience. J Am Coll Radiol 2021;
18:1510–1513
42. Sprague BL, Conant EF, Onega T, et al.; PROSPR Consortium. Variation in
mammographic breast density assessments among radiologists in clini-
cal practice: a multicenter observational study. Ann Intern Med 2016;
165:457–464
43. Lehman CD, Yala A, Schuster T, et al. Mammographic breast density as-
sessment using deep learning: clinical implementation. Radiology 2019;
290:52–58
44. Gastounioti A, Pantalone L, Scott CG, et al. Fully automated volumetric
breast density estimation from digital breast tomosynthesis. Radiology
2021; 301:561–568
45. FDA website. Approval document for DM-Density. www.accessdata.fda.
gov/cdrh_docs/pdf17/K170540.pdf. Published February 23, 2018. Ac-
cessed September 26, 2021
46. FDA website. Approval document for densitas densityai. www.­accessdata.
fda.gov/cdrh_docs/pdf19/K192973.pdf. Published February 19, 2020. Ac-
cessed September 26, 2021
47. FDA website. Approval document for DenSeeMammo. www.accessdata.
fda.gov/cdrh_docs/pdf17/K173574.pdf. Published July 26, 2018. Accessed
September 26, 2021
48. FDA website. Approval document for Insight BD. www.accessdata.fda.
gov/cdrh_docs/pdf17/K172832.pdf. Published February 6, 2018. Accessed
September 26, 2021
49. FDA website. Approval document for PowerLook Density Assessment
V4.0. www.accessdata.fda.gov/cdrh_docs/pdf21/K211506.pdf. Published
July 12, 2021. Accessed September 26, 2021
50. FDA website. Approval document for Quantra. www.accessdata.fda.gov/
cdrh_docs/pdf16/K163623.pdf. Published October 20, 2017. Accessed
September 26, 2021
51. FDA website. Approval document for Visage Breast Density. www.­
accessdata.fda.gov/cdrh_docs/pdf20/K201411.pdf. Published January 29,
2021. Accessed September 26, 2021
52. FDA website. Approval document for Volpara Imaging Software. www.
accessdata.fda.gov/cdrh_docs/pdf21/K211279.pdf. Published July 27,
2021. Accessed September 26, 2021
53. FDA website. Approval document for WRDensity by Whiterabbit.ai. www.
accessdata.fda.gov/cdrh_docs/pdf20/K202013.pdf. Published October
30, 2020. Accessed September 26, 2021
54. Sickles EA, D’Orsi CJ, Bassett LW, et al. ACR BI-RADS Mammography, 5th
ed. In: D’Orsi CJ, Sickles EA, Mendelson EB, et al. ACR BI-RADS Atlas, Breast
Imaging Reporting and Data System. American College of Radiology, 2013
55. Astley SM, Harkness EF, Sergeant JC, et al. A comparison of five methods
AJR:219, September 2022
of measuring mammographic density: a case-control study. Breast Cancer
Res 2018; 20:10
56. Brandt KR, Scott CG, Ma L, et al. Comparison of clinical and automated
breast density measurements: implications for risk prediction and sup-
plemental screening. Radiology 2016; 279:710–719
57. Commonwealth of Massachusetts website. Guidelines for medical neces-
sity determination for breast MRI. www.mass.gov/doc/breast-mri/­
download. Published June 9, 2016. Accessed October 15, 2021
58. Kim G, Bahl M. Assessing risk of breast cancer: a review of risk prediction
models. J Breast Imaging 2021; 3:144–155
59. Brentnall AR, Cuzick J, Buist DSM, Bowles EJA. Long-term accuracy of
breast cancer risk assessment combining classic risk factors and breast
density. JAMA Oncol 2018; 4:e180174
60. Abdolell M, Payne JI, Caines J, et al. Assessing breast cancer risk within the
general screening population: developing a breast cancer risk model to
identify higher risk women at mammographic screening. Eur Radiol 2020;
30:5417–5426
61. Bahl M. Harnessing the power of deep learning to assess breast cancer
risk. Radiology 2020; 294:273–274
62. FDA website. Guidance document: clinical decision support software. https:/​
/www.fda.gov/regulatory-information/search-fda-guidance-­documents/​
clinical-decision-support-software. Published September 27, 2019. Ac-
cessed October 1, 2021
63. Eriksson M, Czene K, Strand F, et al. Identification of women at high risk of
breast cancer who need supplemental screening. Radiology 2020;
297:327–333
64. Strohm L, Hehakaya C, Ranschaert ER, Boon WPC, Moors EHM. Implemen-
tation of artificial intelligence (AI) applications in radiology: hindering and
facilitating factors. Eur Radiol 2020; 30:5525–5532
65. Allen B, Agarwal S, Coombs L, Wald C, Dreyer K. 2020 ACR Data Science
Institute artificial intelligence survey. J Am Coll Radiol 2021; 18:1153–1159
66. Huisman M, Ranschaert E, Parker W, et al. An international survey on AI in
radiology in 1,041 radiologists and radiology residents part 1: fear of re-
placement, knowledge, and attitude. Eur Radiol 2021; 31:7058–7066
67. Chan HP, Samala RK, Hadjiiski LM. CAD and AI for breast cancer—recent
development and challenges. Br J Radiol 2020; 93:20190580
68. Pierce JD, Rosipko B, Youngblood L, Gilkeson RC, Gupta A, Bittencourt LK.
Seamless integration of artificial intelligence into the clinical environ-
ment: our experience with a novel pneumothorax detection artificial in-
telligence algorithm. J Am Coll Radiol 2021; 18:1497–1505
69. Allen B, Dreyer K, Stibolt R Jr, et al. Evaluation and real-world performance
monitoring of artificial intelligence models in clinical practice: try it, buy it,
check it. J Am Coll Radiol 2021; 18:1489–1496
70. Hologic website. Kshirsagar A, Keller B, Smith A. Genius AI Detection for
breast tomosynthesis. www.hologic.com/sites/default/files/2020_12/
WP-00178_Rev02_GeniusAI_Detection-white-paper-6979r10p.pdf. Pub-
lished December 2020. Accessed October 20, 2021
71. Kshirsagar M, Robinson C, Yang S, et al. Becoming good at AI for good.
arXiv website. arxiv.org/abs/2104.11757. Published May 3, 2021. Accessed
October 22, 2021
72. Spilseth B, McKnight CD, Li MD, et al. AUR-RRA review: logistics of aca-
demic-industry partnerships in artificial intelligence. Acad Radiol 2022;
29:119–128
73. Ryan ML, O’Donovan T, McNulty JP. Artificial intelligence: the opinions of
radiographers and radiation therapists in Ireland. Radiography (Lond)
2021; 27(suppl 1):S74–S82
74. U.S. Congress website. H.R.34: 21st Century Cures Act. www.congress.
gov/bill/114th-congress/house-bill/34. Published December 13, 2016. Ac-
379
 Lamb et al.
cessed October 15, 2021
75. FDA website. How to determine if your product is a medical device. www.
fda.gov/MedicalDevices/DeviceRegulationandGuidance/Overview/­Classify​
YourDevice/ucm051512.htm. Published December 16, 2019. Accessed Oc-
tober 15, 2021
76. FDA website. Learn if a medical device has been cleared by FDA for mar-
keting. www.fda.gov/medical-devices/consumers-medical-devices/learn-​
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
if-medical-device-has-been-cleared-fda-marketing. Published Decem-
ber 29, 2017. Accessed October 15, 2021
77. Kaushal A, Altman R, Langlotz C. Geographic distribution of U.S. cohorts
used to train deep learning algorithms. JAMA 2020; 324:1212–1213
78. Geis JR, Brady AP, Wu CC, et al. Ethics of artificial intelligence in radiology:
summary of the joint European and North American multisociety state-
ment. Radiology 2019; 293:436–440
79. Bero L. Addressing bias and conflict of interest among biomedical re-
searchers. JAMA 2017; 317:1723–1724
80. Ongena YP, Yakar D, Haan M, Kwee TC. Artificial intelligence in screening
mammography: a population survey of women’s preferences. J Am Coll
Radiol 2021; 18(1 Pt A):79–86
81. Geras KJ, Mann RM, Moy L. Artificial intelligence for mammography and
digital breast tomosynthesis: current concepts and future perspectives.
Radiology 2019; 293:246–259
82. Geras KJ, Wolfson S, Shen Y, et al. High-resolution breast cancer screening
with multi-view deep convolutional neural networks. arXiv website. arxiv.
org/abs/1703.07047. Published June 28, 2018. Accessed October 28, 2021
83. Wu N, Jastrzębski S, Park J, Moy L, Cho K, Geras KJ. Improving the ability of
deep neural networks to use information from multiple views in breast
cancer screening. Proc Mach Learn Res 2020; 121:827–842
84. Lotter W, Diab AR, Haslam B, et al. Robust breast cancer detection in mam-
mography and digital breast tomosynthesis using an annotation-efficient
deep learning approach. Nat Med 2021; 27:244–249
85. American College of Radiology Data Science Institute website. ACR
AI-LAB. ailab.acr.org/Account/Home. Accessed December 1, 2021
86. Gastounioti A, Kontos D. Is it time to get rid of black boxes and cultivate
trust in AI? Radiol Artif Intell 2020; 2:e200088
87. Hickman SE, Baxter GC, Gilbert FJ. Adoption of artificial intelligence in
breast imaging: evaluation, ethical constraints and limitations. Br J Cancer
2021; 125:15–22
380
88. American College of Radiology Data Science Institute website. Define-AI Di-
rectory. www.acrdsi.org/DSI-Services/Define-AI. Accessed December 1, 2021
89. Cai H, Huang Q, Rong W, et al. Breast microcalcification diagnosis using
deep convolutional neural network from digital mammograms. Comput
Math Methods Med 2019; 2019:2717454
90. CureMetrix website. cmAngio. curemetrix.com/cmangio/. Accessed De-
cember 8, 2021
91. Yala A, Lehman C, Schuster T, Portnoi T, Barzilay R. A deep learning mam-
mography-based model for improved breast cancer risk prediction. Ra-
diology 2019; 292:60–66
92. Yala A, Mikhael PG, Strand F, et al. Toward robust mammography-based
models for breast cancer risk. Sci Transl Med 2021; 13:eaba4373
93. Rodríguez-Ruiz A, Lång K, Gubern-Merida A, et al. Can we reduce the
workload of mammographic screening by automatic identification of
normal exams with artificial intelligence? A feasibility study. Eur Radiol
2019; 29:4825–4832
94. Yala A, Schuster T, Miles R, Barzilay R, Lehman C. A deep learning model to
triage screening mammograms: a simulation study. Radiology 2019;
293:38–46
95. Langlotz CP, Allen B, Erickson BJ, et al. A roadmap for foundational re-
search on artificial intelligence in medical imaging: from the 2018 NIH/
RSNA/ACR/The Academy Workshop. Radiology 2019; 291:781–791
96. Allen B Jr, Seltzer SE, Langlotz CP, et al. A road map for translational re-
search on artificial intelligence in medical imaging: from the 2018 Nation-
al Institutes of Health/RSNA/ACR/The Academy Workshop. J Am Coll Radi-
ol 2019; 16(9 Pt A):1179–1189
97. Sogani J, Allen B Jr, Dreyer K, McGinty G. Artificial intelligence in radiolo-
gy: the ecosystem essential to improving patient care. Clin Imaging 2020;
59:A3–A6
98. Chen MM, Golding LP, Nicola GN. Who will pay for AI? Radiol Artif Intell
2021; 3:e210030
99. Ong MS, Mandl KD. National expenditure for false-positive mammograms
and breast cancer overdiagnoses estimated at $4 billion a year. Health Aff
(Millwood) 2015; 34:576–583
100. van Winkel SL, Rodríguez-Ruiz A, Appelman L, et al. Impact of artificial in-
telligence support on accuracy and reading time in breast tomosynthesis
image interpretation: a multi-reader multi-case study. Eur Radiol 2021;
31:8682–8691
(Editorial Comment starts on next page)
AJR:219, September 2022
 AI for Screening Mammography
Editorial Comment: Artificial Intelligence in Mammography—Our New Reality
In this rapidly evolving environment of artificial intelligence
(AI), mammography has been a prime target for machine learn-
ing, particularly deep learning (DL), applications. Historically,
breast imaging has been one of the most commonly targeted
imaging applications for computer-based detection and classi-
fication systems [1]. However, computer-aided detection (CAD)
Downloaded from ajronline.org by 85.107.71.160 on 03/24/24 from IP address 85.107.71.160. Copyright ARRS. For personal use only; all rights reserved
showed limited value in screening mammography [2]. The prior
failure of CAD in mammography was attributed predominantly
to supervised learning and insufficient processing power [3]. In
comparison, modern DL overcomes these problems through the
currently available tremendous computing power and through
self-identification of important imaging features.
There has been a spurt of mammography AI products in the mar-
ketplace recently. The authors of this study provide a comprehensive
overview of commercially available AI products for lesion detection
and diagnosis in screening mammography, triage of mammograms,
breast density assessment, breast cancer risk assessment, and im-
provements in image quality and patient positioning. They also de-
scribe the diagnostic accuracy of AI algorithms as reported in the lit-
erature and explore issues related to clinical implementation.
Although many mammography AI products have transitioned
from research to clinical practice, questions remain about the
true impact on patient outcomes, as most studies of these algo-
rithms have been multireader studies of cancer-enriched data-
sets, which is not representative of real-world practice. Lack of
clinical validation and the need for better integration into the
AJR:219, September 2022
workflow are immediate challenges that these products face as
they transition from research to clinical practice. Medicolegal is-
sues, governance and ethical issues, and the cost-benefit prop-
osition must also be addressed, not just for mammography AI
products but for all AI-based imaging products. Nonetheless, as
AI in mammography is now a reality, breast imaging radiologists
must have an understanding of the available products and their
clinical utility. This article provides such an overview.
Sadia Khanani, MD
Mayo Clinic
Rochester, MN
khanani.sadia@mayo.edu
S. Khanani has received research funding from Siemens Healthineers, and S. Khanani
and Mayo Clinic have a research and development agreement with Imago Systems.
doi.org/10.2214/AJR.22.27345
References
1. Giger ML, Chan HP, Boone J. Anniversary paper: history and status of CAD
and quantitative image analysis: the role of medical physics and AAPM.
Med Phys 2008; 35:5799–5820
2. Lehman CD, Wellman RD, Buist DS, Kerlikowske K, Tosteson AN, Miglioretti
DL; Breast Cancer Surveillance Consortium. Diagnostic accuracy of digital
screening mammography with and without computer-aided detection.
JAMA Intern Med 2015; 175:1828–1837
3. Kohli A, Jha S. Why CAD failed in mammography. J Am Coll Radiol 2018; 15(3
Pt B):535–537
381