Journal of the Association of American Medical Colleges

Uncomposed, edited manuscript published online ahead of print.

This published ahead-of-print manuscript is not the final version of this article, but it may be cited and
shared publicly.

D
Author: Gesundheit Neil MD, MPH

TE
Title: A Crisis of Trust Between U.S. Medical Education and the National Board of Medical Examiners

DOI: 10.1097/ACM.0000000000003131
EP
C
C
A

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Academic Medicine

DOI: 10.1097/ACM.0000000000003131

A Crisis of Trust Between U.S. Medical Education and the National Board of Medical

Examiners

Neil Gesundheit, MD, MPH

D
N. Gesundheit is professor of medicine and senior associate dean for medical education,

Stanford University School of Medicine, Stanford, California.

TE
Correspondence should be addressed to N. Gesundheit, Stanford School of Medicine,

Educational Programs and Services, 1265 Welch Rd., MSOB x321, Stanford CA 94305-5404;

telephone: (650) 724-5454; email: neil7@stanford.edu.

Editor’s note: This is an Invited Commentary on Carmody JB, Rajasekaran SK. On Step 1
EP
mania, USMLE score reporting, and financial conflict of interest at the National Board of

Medical Examiners. Acad Med. XXX.

Funding/Support: None reported.

Other disclosures: None reported.

C

Ethical approval: Reported as not applicable.

Disclaimer: The comments represent the views of the author and not necessarily those of
C

Stanford University.
A

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Abstract

This Invited Commentary is an independent opinion piece and companion to the Perspective by

Carmody and Rajasekaran that appears in this issue of Academic Medicine. The National Board

of Medical Examiners (NBME), a 501(c)(3) nonprofit, is a powerful gatekeeper to the medical

profession in the United States. According to publicly available tax data, the NBME, which has

D
increased its number of income-enhancing products, had revenues of $153.9 million (M) and net

assets of $177.6M in 2017, earnings (revenue less expenses) of $39.7M in 2013-17, and a highly

TE
compensated management team. Medical students are ultimately the source of nearly all the

NBME’s revenue, and the NBME has contributed to the growth of medical student debt. The

NBME has operated as a monopoly since its agreement in the early 1990s with the Federation of

State Medical Boards to co-sponsor the United States Medical Licensing Examination (USMLE).
EP
Although the NBME has developed valuable products and is ostensibly governed by a capable

board, the NBME has inherent financial conflicts of interest and may be benefiting from the

current “Step 1 mania” undermining undergraduate medical education. Here, the author makes 4

recommendations to reestablish the trust of the U.S. medical education community in the
C

NBME: (1) the NBME should recuse itself from current discussions and policy-making decisions

related to changes in the score reporting of the USMLE Step 1 exam; (2) the NBME should
C

disclose and be transparent about all aspects of its finances; (3) new NBME products, changes in

pricing, and changes to pass thresholds should be approved by an oversight committee,

A

independent of the NBME; and (4) the NBME (and USMLE) should not charge students or

residents for re-taking any of its licensing examinations.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 The National Board of Medical Examiners (NBME) is a powerful, well-funded organization and

gatekeeper to the medical profession; it sells products and services and provides state medical

boards with a common system for licensing physicians in the United States. Here, I comment on

the Perspective by Carmody and Rajasekaran in this issue of Academic Medicine that explores

whether the NBME’s score-reporting policy, along with its current products and services,

D
constitutes a conflict of interest.1 Specifically, I explore the history and more recent financial

growth of the NBME; I argue that the NBME constitutes a self-regulated monopoly; I examine

TE
the influence of what Carmody and Rajasekaran characterize as “Step 1 mania”; and I outline 4

steps the NBME can take to regain the trust of the medical education community.

The Origins and Evolution of the NBME and USMLE

Founded 5 years after the Flexner Report of 1910, the NBME was established with the noble
EP
goal of creating a rational, standardized system by which physicians could demonstrate their

knowledge and suitability to practice medicine. The underlying goal of the NBME was to serve

the public and ensure that practicing physicians in the United States met standards of

competency in medical knowledge.

C

For its first 75 years, the NBME competed with an alternate pathway to licensure, the Federation

Licensing Exam (FLEX) administered by the Federation of State Medical Boards (FSMB). In
C

1992, the NBME and FSMB engaged in a collaborative agreement to cosponsor and unify their

licensure exams in the form of the United States Medical Licensing Examination (USMLE). The
A

USMLE initially comprised 3 parts, and in 2004 the NBME and FSMB added a fourth clinical

competency component, Step 2-CS (clinical skills). The NBME has primary oversight of Step 1

and Step 2-CK (clinical knowledge); the organizations share oversight of Step 2-CS; and the

FSMB has primary oversight of Step 3.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 The Financial Growth of the NBME: A Nonprofit with Growing Revenues, Margins, and

Net Assets

Organizations, such as the NBME, that receive 501(c)(3) status under the U.S. Internal Revenue

Code are considered nonprofits and, as such, do not have owners or shareholders and cannot

distribute earnings; however, these registered nonprofits are often encouraged by their governing

D
boards and market forces to grow their revenues and increase net assets.

The NBME reports its financial status to the Internal Revenue Service (IRS) annually on IRS

TE
Form 990 (the nonprofit equivalent of IRS Form 1120 for corporations and Form 1040 for

individuals). Although financial details of the NBME’s operations are not provided in the

organization’s 2018 annual report,2 IRS Tax Forms 990 are available to the public.3 In 2017 the

NBME reported program service revenue of $153.9 million (M) and earnings (revenue less
EP
expenses) of $8.0M.3 Over the 5-year period from 2013 to 2017, the NBME’s overall program

service revenue was $712.1M, with an annual growth rate of 5% per year.3 The aggregate

earnings during this 5-year period were $39.7M.3 The NBME’s profit margin ratio, defined as net

income (revenue less expenses) divided by net sales (program service revenue) over this 5-year
C

period was approximately 6%. For comparison purposes, the margins of the for-profit companies

Walmart, Target, and Ford Motor Company over this same period averaged, respectively, 3%,
C

4%, and 4%.4 Because of these years of positive revenue and robust margins, the NBME’s net

assets grew to $177.6M at the end of tax year 2017.3

A

Carmody and Rajasekaran have dug deeper into NBME’s finances and have shown that the

organization’s revenues more than tripled between 2001 and 2017, far outpacing inflation.1 The

fee for Step 1, a main revenue generator, increased from $435 in 2004 to $630 in 2019, again

exceeding the rate of inflation.1 Further from 2008 to 2017 the NBME workforce grew to include

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 more than 200 employees earning more than $100,000 per year.1 In 2017, the NBME’s retiring

president received over $1M, and the incoming president received over $820,000, in total

compensation.3 Fourteen employees earned more than $400,000.3 This compensation for top

NBME executives in 2016-17 was easily in the top 0.5% of charitable nonprofits, according to

Charity Navigator (charitynavigator.org), an organization that tracks the salaries of nonprofit

D
CEOs.5(pp9-10)

In many ways, the NBME has become a big business, with growing revenues of more than

TE
$700M over the 5-year period ending in 2017, new products, positive earnings of nearly $40M

during this same period, strong margins, and highly paid staff and executives. While this growth

and financial health is in many ways to the credit of NBME management, the figures also

highlight the inherent risk of placing the organization at the nexus of key decision-making
EP
regarding requirements for the use of its products.

The High NBME Fees Increase the Cost of Medical Education and Medical Student Debt

Notably, the consumers of NBME services and products are medical students, which as Carmody

and Rajasekaran outline, has consequences for the students themselves and medicine.1 Medical
C

students ultimately pay the cost of nearly all of NBME services, in direct and indirect charges.

The direct cost for U.S. applicants who take the USMLE is $630 for each step exam: Step 1 and
C

Step 2-CK. Costs are approximately 50% higher for international medical graduates. Students

incur further direct costs when they purchase NBME practice exams. Because the Step 1 and
A

Step 2-CK tests have become high-stakes exams, most medical students take NBME-sponsored

practice exams—the Comprehensive Basic Science Self-Assessment (CBSSA) for Step 1, and

the Comprehensive Clinical Science Self-Assessment Exam (CCSSA) for Step 2-CK—at a cost

of $60 per exam. The number of CBSSA exams purchased by medical students grew 10-fold

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 between 2005 and 2016.1 With the growth of Step 1 mania, practice exams have become one of

the largest areas of product and profit margin growth at the NBME.1 Because of the popularity of

practice exams, the NBME constantly refreshes its offerings (“forms”), ensuring strong revenues

for this product. Although anxiety about Step 1 is well established, anxiety over Step 2-CK has

also been increasing, and medical students are purchasing increasing numbers of CCSSA tests,

D
also at a cost of $60 per test. Assuming medical students each purchase a total of 5 Step 1 and

Step 2-CK practice exams, then additional expenses incurred equal $300 per applicant. The

TE
additional cost of Step 2-CS ($1,290) and Step 3 ($875) bring the total expenses incurred directly

by students for USMLE services to $3,725 (which does not include expenses for traveling to the

Step 2-CS test site). These expenses do not include supplementary preparatory (“test prep”)

materials, which are also in common use.

EP
There are also indirect costs to medical students. Most medical schools require students to take

NBME knowledge (shelf) exams during their clinical training. The cost of these exams to

medical schools is approximately $46 per exam per student. Further, assuming that students take

at least 5 shelf exams during medical school, the cost for this additional testing, when passed on
C

as student fees or tuition, is at least $230 per student. Students may bear the burden of additional

expenses incurred by medical schools that use the NBME’s Customized Assessment Services
C

(CAS); this service allows schools to create exams from the USMLE question bank.

Not including expenses for CAS, the total fees for USMLE services are approximately $4,000
A

(including both indirect and direct costs) per U.S. medical student.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 To summarize thus far, the cost of USMLE assessments is significant, particularly in light of the

high and rapidly rising medical student debt, which now approaches a median of $200,000.6 The

NBME, while registered as a 501(c)(3) nonprofit, enjoys large revenues and strong margins,

employs a highly paid executive staff, and charges significant fees that contribute to medical

student debt.

D
The NBME is a Self-Regulated Monopoly

As mentioned, prior to 1992-94, most states accepted two pathways for medical licensure, the

TE
NBME and the FLEX. The 1992 agreement to merge licensure testing created a monopoly—that

is, a commodity or service in exclusive control by a company or group. Most public-service

monopolies, whether for-profit or nonprofit, are regulated by an outside, impartial board that

safeguards the public trust against the risk of abuse. As an example, electricity and natural gas
EP
services in southern California are provided nearly exclusively by the Southern California Edison

(SCE) Company. Any change in pricing for SCE’s services must be approved by the California

Public Utilities Commission (CPUC). CPUC commissioners are appointed by the governor and

confirmed by the California State Senate. When SCE registers to raise its rates for electricity, it
C

must document, for the CPUC, the greater cost of generating electricity or purchasing natural

gas, any alternative sourcing that it has sought, and the effect of the price increase on
C

communities. Public hearings often take place wherein the company faces scrutiny and must

defend its pricing decisions. CPUC commissioners then vote to approve or reject SCE’s
A

proposals. In short, neither SCE nor its board unilaterally set SCE’s rates.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 By contrast, the NBME is regulated by an internal board, and changes in pricing strategies or

new products do not require approval by a public group or an indisputably independent body,

such as the Liaison Committee on Medical Education of the Association of American Medical

Colleges (AAMC) or the AAMC’s Council of Deans. Although the NBME has a sizeable and

diverse 80-member board, it is not clear whether changes in pricing structures are openly

D
discussed and debated beyond perhaps closed meetings of the NBME Executive Board (of

approximately 10 members). In addition, the expertise of the members of the NBME Executive

TE
Board is primarily education. In contrast to a public utility that is prevented from monopolistic

abuses by external regulation, few checks and balances help control and regulate the NBME. The

price charged for Step 1 and Step 2-CK examinations, as an example, can be raised without

extensive justification or the scrutiny of the medical education community. The NBME can also
EP
raise its passing cut-off score for Step 1 and Step 2-CK without strong, evidence-based

justification that a higher number predicts greater clinical success. More exam failures generate

more revenue both from re-testing and from increased sales of practice materials to students

hoping to surpass the higher bar (see below). While it may not be the organization’s intent, the
C

NBME benefits whenever pass thresholds increase.

Given that the NBME benefits from re-testing and test preparation materials, its board members
C

are likely not objective participants or arbiters of the current Step 1 score reporting debate. The

organization’s desire for growth and influence may be jeopardizing its original noble mission
A

and its source of influence.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 The NBME Is a Registered Nonprofit, But It Is Not Charitable

Perhaps equally distressing to medical students as the cost of USMLE testing is that the NBME

and FSMB charge full fees to students who fail and must re-take any of the 4 USMLE exams.

This policy of charging a re-testing fee is in line with the policy of the American Board of

Medical Specialties, but there is a key difference. Medical specialists who take board exams are

D
licensed physicians on the threshold of earning average salaries of approximately $300,000

annually.7 Medical students taking USMLE exams have a mean debt of $200,0006 and are on the

TE
threshold of 3 to 7 years of training before they are able to begin to pay off debt. Thus, in

addition to the humiliation, psychological damage, and stigma of failing a licensing exam, a

medical student bears the added burden of paying a second test fee. Given my years working

with students, I can attest to the financial struggles students face. One medical student I
EP
mentored required 3 attempts before passing Step 2-CS and incurred $4,000 of debt in the

process. Surely the NBME and FSMB, with significant net assets and an IRS classification in a

category that includes charities, can be more charitable in their treatment of physicians in

training.
C

The NBME and Step 1 Mania—A Perfect Toxic Storm

National debate about the disproportionate emphasis placed by graduate medical education
C

(GME) program directors (PDs) on the USMLE Step 1 exam has been accelerating, especially

given the negative consequences not only on medical students and their mental health but also on
A

undergraduate medical education (UME).8 As the primary sponsor of Step 1, the NBME will

undoubtedly bear financial consequences for any change to the format or grading of the Step 1

exam; thus, how the NBME interacts with the decision to change Step 1 score reporting is a

relevant concern for medical educators.

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Some of the concerns about the Step 1 exam and Step 1 mania have been discussed previously in

this journal8,9 and in other forums. The dilemma has 3 basic aspects:

1. The Step 1 exam is used by many residency fields and PDs as a gatekeeper or filter, even

though it was designed as a pass/fail exam and has no rigorous data to validate its use to

predict residency performance;10

D
2. Because of the way many residency PDs use Step 1, the exam is perceived by students as

the “ticket” to residency; and

TE
3. Because of this perception, many medical students make achieving a high Step 1 score

their foremost objective for their pre-clerkship studies. They downgrade other topics and

activities (e.g., understanding disease mechanisms; enhancing critical thinking and

clinical reasoning; improving physical examination, interview, and communication skills;

EP
understanding the psycho-social context of illness; undertaking early clinical experiences

and community engagement; volunteering; maintaining physical and emotional health

and healthy relationships) as they, instead, elevate Step 1.

At Stanford, despite a faculty with 8 Nobel laureates and extensive clinical facilities that include
C

5 teaching hospitals, some medical students virtually ignore the pre-clerkship curriculum in favor

of one or more parallel curricula they create on their own. Their teachers are First Aid
C

(https://firstaidteam.com/; published by McGrawHill Education), U World

(https://www.uworld.com/; Dallas, Texas), Pathoma (https://www.pathoma.com/; Chicago,

A

Illinois), Sketchy Medical (https://sketchymedical.com/; Los Angeles, California), or one of 20

additional resources designed to help students maximize their Step 1 score.8 As the Step 1 exam

approaches, classrooms and voluntary laboratory exercises are lightly attended. Students

decrease interactions with one another. Community clinics, normally supported by medical

10

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 students, are understaffed. Moreover, although this parallel curriculum no doubt has some merit,

many of the techniques it inculcates are aimed to improve factual recall rather than

understanding; that is, as Wartman has described, they focus on building “information” rather

than the higher goal of building “knowledge.”11 Information is easily and reliably obtained in

clinical practice by using a search tool on a smart phone, laptop, or portable tablet computer.

D
Why graduate students would forego an in-depth and multidimensional medical curriculum that

helps them achieve knowledge and an understanding of medicine for, instead, learning tools that

TE
primarily encode memorization of facts might seem inexplicable. But students do, and the reason

is Step 1 mania and the toxic storm it has created.

The NBME, whether by intent or happenstance, has been the beneficiary of Step 1 mania. With

the increasing pressure to perform well on Step 1, students have used NBME practice exams and
EP
resources to boost their Step 1 scores. In response to rising national scores, which are in turn the

result of better espionage by the burgeoning Step 1 prep industry, the NBME has asked question

developers to create new material, often resulting in the testing of obscure and less relevant facts.

This pursuit of medical trivia has affected medical education. For example, in the endocrine
C

physiology course that I have taught for 20 years, when my teaching colleagues and I discuss

hormonal signaling, we review the mechanisms and diversity of signal transduction and the ways
C

that it can be modulated and disrupted. We discuss more than 50 hormones, but emphasize that

memorizing exactly how each hormone signals is not important since specifics can always be
A

looked up if they become relevant to patient care or research. Yet students report that our advice

has been misguided; the Step 1 exam, in some iterations, has indeed tested for detailed facts

about specific hormone signaling mechanisms. This drive for detail appears to define the

“excellent” candidate for PDs who use Step 1 as a guide to identifying trainees worthy of

11

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 acceptance into their residency programs. Thus, the Step 1 exam has badly distorted the priorities

of U.S. medical students. Assessment drives curriculum (even if inadvertently),12 and the current

Step 1 climate has had a severe and adverse effect on curriculum development and student-

faculty engagement.

As mentioned, this issue of Academic Medicine features a timely exposé by Carmody and

D
Rajasekaran.1 The InCUS (Invitation Conference on USMLE scoring) meeting held in March

2019 was an opportunity to discuss the topic of Step 1 score reporting (Pass/Fail vs. numerical

TE
score) and its effect on students and medical education, but the fact that the NBME and FSMB

were organizers at the epicenter of the meeting undermined its legitimacy. I agree with Carmody

and Rajasekaran: the NBME has a deep and inescapable financial conflict of interest in

determining the future direction of the Step 1 exam. Below, I outline 4 steps, including recusal
EP
from discussion and decision-making on the topic of Step 1 score reporting, that the NBME can

take to regain the trust of the medical education community.

A Proposal With Four Steps to Regain Trust in the NBME

The 4 steps I propose—recusal, transparent financial reporting, an independent oversight

C

committee, and free retesting—will implement checks and balances with regard to the activities

of the NBME and will begin the process of restoring confidence in the organization.
C

1. The NBME should recuse itself from current discussions and decision-making about the

reorganization of the Step 1 Exam

A

The NBME may feel that it “owns” the Step 1 exam, but medical educators (including UME

faculty and GME PDs), students (future physicians), and state medical boards should be the true

owners of the exam. The NBME’s primary role is to implement a licensing exam to safeguard

the public, not to influence medical education. Because of the NBME’s conflicts of interest, as

12

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 described by Carmody and Rajasekaran1 and with which I agree, the NBME should recuse itself

from participating in discussions and decision-making about changes to the grade reporting,

minimum passing score, and other issues surrounding Step 1.

2. The NBME must be transparent in disclosing its full finances

The NBME’s annual reports must show the organization’s full finances, including the sources

D
and amounts of revenues and expenses. The NBME should disclose salaries paid to the most

highly compensated employees, which can be shown in an anonymized format. Annual reports

TE
must also describe the decisions made by the NBME’s Compensation Committee. This

information is currently missing from its 2018 56-page annual report.2 For-profit corporations

regularly provide this information either in their annual reports or in quarterly 10-K forms. The

NBME’s size and importance to medical education demand a higher level of transparency than
EP
the organization has heretofore provided. Disclosing the information on annual IRS Form 990 is

required by law, but it is hidden from easy public view and, thus, is not sufficient.

3. New products, changes in prices for current products, and changes to pass thresholds

must be approved by an independent oversight committee

C

The NBME provides required services for which there is no competition; therefore, changes to

its products and pricing should be approved by an independent oversight committee. Currently
C

the NBME maintains a board comprised of approximately 80 diverse and talented members,

mostly medical educators, who represent many constituencies and provide vision for the
A

organization’s activities.2 An executive board of approximately 10 members is elected from the

larger board.2 This system provides the appearance of independence, but NBME management

may still exert influence in the selection of some board members. Despite the talented

constituents of the NBME board and its smaller executive board, the AAMC or another suitable

13

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 third-party should organize a financial oversight committee that is unquestionably and fully

independent of the NBME. Any new product, change in pricing, or change in pass threshold of

an NBME licensing exam should be approved by such an independent oversight committee—just

as an oversight committee reviews and approves changes to prices charged by a public utility

company. Similarly, given the NBME’s status as sole provider of required services, the

D
committee members must be appointed through an unquestionably independent process.

4. The NBME should not charge a fee for retaking a failed licensing exam

TE
Failure on a licensing exam achieves its purpose—to identify a student who needs to improve

knowledge and/or skills in the tested area—but recognizing that students suffer humiliation and

psychological trauma when they fail an exam is important. Adding a steep re-testing fee is

unnecessary and compounds harm incurred by a student who is likely already in debt. Given the
EP
NBME’s considerable net assets of nearly $180M and positive earnings of $40M in 2013-17, this

re-test fee should be waived.

In Sum

The NBME, although registered as a 501(c)(3) nonprofit, has grown to become a powerful
C

organization that resembles a for-profit corporation. It enjoys significant revenue and earnings,

has increased its offerings of revenue-enhancing products, and employs a highly compensated
C

executive management team. Medical students are ultimately the source of nearly all the

NBME’s revenue. The growth of the NBME has coincided with and contributed to the increase
A

in medical student debt, which is at historically high levels. The NBME has operated essentially

as a monopoly since its agreement in the early 1990s with the FSMB to co-sponsor the 4-part

USMLE, which is required for licensing physicians in the United States. Although the NBME

has developed many valuable products and is ostensibly governed by a talented and capable

14

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 board, the NBME has inherent financial conflicts of interest, appears to profit from its status as

gatekeeper to the medical profession, and should, therefore, recuse itself (or be forced to be

recused) from discussions or policy-making decisions related to changing the grade reporting of

(and other aspects related to) the USMLE Step 1 exam. Further, the NBME should be transparent

in all aspects of its finances, and new NBME products and pricing structures should be approved

D
by an independent oversight entity. Finally, the NBME should not charge medical students and

residents to re-take USMLE licensing exams in cases of failure.

TE
Although this Invited Commentary and the accompanying article by Carmody and Rajasekaran

have addressed concerns about the NBME,1 similar concerns likely apply to the NBME’s partner

organization, the FSMB. Next steps should include scrutinizing its practices and any real or

apparent conflicts of interest.

EP
C
C
A

15

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 References

1. Carmody JB, Rajasekaran SK. On Step 1 mania, USMLE score reporting, and financial

conflict of interest at the National Board of Medical Examiners. Acad Med. XXX.

2. National Board of Medical Examiners. Engagement and Action: 2018 Annual Report.

Philadelphia: National Board of Medical Examiners.

D
https://www.nbme.org/PDF/Publications/2018Annual-Report.pdf. Accessed November 27,

2019.

TE
3. Cause IQ. Cause IQ Organization Profile: National Board of Medical Examiners.

https://www.causeiq.com/organizations/national-board-of-medical-examiners,231352238/.

Accessed November 27, 2019.

4. Macrotrends. Profit Margin 2006-2019.

EP
https://www.macrotrends.net/stocks/charts/WMT/walmart/profit-margins. Accessed

December 10, 2019.

5. Charity Navigator. 2016 Charity CEO Compensation Study. October 2016.

https://d20umu42aunjpx.cloudfront.net/2016+CEO+Comp+Study/2016+CEO+Compensatio
C

n+Study.pdf. Accessed November 27, 2019.

6. Association of American Medical Colleges. Education Debt Manager for Matriculating and
C

Graduating Medical School Students, 2019.

https://store.aamc.org/downloadable/download/sample/sample_id/260/. Accessed November

A

27, 2019.

7. O’Connell B. How much do doctors make in 2018? TheStreet. November 13, 2018.

https://www.thestreet.com/personal-finance/how-much-do-doctors-make-14779617.

Accessed November 27, 2019.

16

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 8. Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives

on the “Step 1 climate” in preclinical medical education. Acad Med. 2019; 94;302-304.

9. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of the United States

Medical Licensing Examination Step 1 scores in residency selection. Acad Med.

2016;91:12-15.

D
10. Hamdy H, Prasad K, Anderson MB, et al. BEME systematic review: Predictive values of

measurements obtained in medical schools and future performance in medical practice. Med

TE
Teach 2006; 28,2:103-116.

11. Wartman SA. The empirical challenge of 21st-century medical education. Acad Med. 2019;

94:1412-1415.

12. Wormald BW, Schoeman S, Somasunderam A, Penn M. Assessment drives learning: An

EP
unavoidable truth? Anat Sci Educ. 2009;2:199-204.
C
C
A

17

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.