ORIGINAL ARTICLE
Alternative Strategies for Evaluating General Surgery Residency
Applicants and an Interview Limit for MATCH 2021
An Impending Necessity
Aashish Rajesh, MBBS,  Y and Malke Asaad, MDy
Downloaded from https://journals.lww.com/annalsofsurgery by 01UGrXh3ipqzR4DKqW7bOJtSxsRVOheLV9OzOeHq2POX2t1GrKUD6m1aBQdlm0lX7bZxxCFBpuxAM3exuxdXARVaPXROhrEIELxLuiE5dAoxEXL1EbBXOmtN5qOe2K0czeIvdGFOHR5ia7TbUFTzsQ== on 03/16/2021
The COVID-19 pandemic has posed unique challenges for evaluating general
surgery residency applicants to MATCH 2021. In the absence of away
rotations, programs are likely to afford greater importance to objective data
to stratify the applicant pool and medical students are likely to experience
difficulty in thoroughly assessing each program. Virtual rotations, meet-and-
greet events conducted before the application submission deadline, personal-
ity testing before extending interviews, standardized letters of recommenda-
tion, and skills testing can serve as valuable adjuncts for determining the best
applicant-program fit. Finally, an interview limit which sets the bar for the
maximum number of accepted interviews per applicant per specialty can offer
a level playing field in the absence of time and cost limitations associated with
travel.
Keywords: away rotations, general surgery, interviews, residency, resident
selection, surgical education
(Ann Surg 2021;273:109–111)
T he COVID-19 pandemic has had a profound impact on medical
student education and has created unique challenges for evalu-
ating prospective residency applicants. The work group established
by the Coalition for Physician Accountability has strongly discour-
aged away rotations for the 2020 to 2021 academic year with the
exception of students lacking specific rotations at their home insti-
tution or those requiring these rotations for graduation.1 In accor-
dance with these guidelines, the Association of Program Directors in
Surgery (APDS) issued a statement on May 19, 2020, recommending
that program directors not consider participation in an away rotation
while evaluating residency applicants.2
Keeping with the norms of social distancing and travel con-
cerns, interviews for the upcoming residency cycle will follow a
virtual format. Albeit advantageous in terms of cost and time, web-
based interviews limit a thorough evaluation of an applicant’s
personality, ability to blend with the program and current residents,
and have been reported to affect applicants’ ability to present
themselves to interviewing faculty.3 All these changes constitute
significant impediments to the assessment of an applicant’s clinical
knowledge and skill, which are otherwise apparent through clinical
rotations, and form basic requirements for future surgical residents.
Through this article, we underscore the traditional importance
attributed to away rotations and propose suggestions to bridge at
From the University of Texas Health Science Center, San Antonio, Texas; and
yM.D. Anderson Cancer Center, Houston, Texas.
ashish747raj@gmail.com; rajesha@uthscsa.edu.
The authors report no conflicts of interest.
Copyright ß 2020 Wolters Kluwer Health, Inc. All rights reserved.
ISSN: 0003-4932/20/27301-0109
DOI: 10.1097/SLA.0000000000004501
Annals of Surgery  Volume 273, Number 1, January 2021
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
least in part, the massive void in applicant evaluation that has resulted
from the effects of the pandemic.
Away Rotations – Importance, Changes, and
Consequences
Audition rotations serve as an opportunity for fourth-year
medical students to complete 2 to 4 weeks at other institutions, to
glean clinical experience and evaluate a prospective residency
program. Likewise, the program faculty are able to assess if the
participating student would blend well as a resident. In general
surgery (GS), data from the National Resident Matching Program
(NRMP) Program Director survey showed that an audition elective
within the department received an average importance rating of 3.9
on a 5.0 scale when selecting an applicant to interview, and a score of
4.1 while ranking applicants.4
Audition rotation experience can also be negative – Higgins
et al found that 53% of their student respondents reported ranking a
program lower after an audition elective.5 With the suspension of
away rotations, the lack of first-hand experience in a particular
residency program culture may manifest as subsequent attrition in
the years to come. Given the prevalence of an almost 20% attrition
rate in GS, it is imperative to develop novel opportunities and virtual
rotations to involve prospective applicants, as a means of determining
a good applicant-program fit.2,6,7
Virtual clinical rotations comprise an evolving concept neces-
sitated by restrictions stemming from the COVID-19 pandemic. As a
framework for such rotations, Calhoun et al described a novel model
which entailed an introduction lecture discussing perioperative sur-
gical care, paired with didactic modules and assessments.8 In 2016,
Sullivan et al from the University of Wisconsin had conducted a pilot
study of virtual surgical patient cases for third-year medical students
together with faculty-facilitated discussions, and determined that
students improved their clinical reasoning skills significantly with
each case attempt.9 Given the present day scenario, adopting similar
models coupled with the engagement of medical students in resident
didactics, and encouraging active student participation through the
utilization of online platforms would likely offer potential applicants
an opportunity to evaluate each program culture and resident train-
ing. Some surgical specialties have started offering such opportu-
nities for medical students, involving live streaming of surgeries,
virtual campus tours, participation in clinic interactions/faculty-led
case discussions, complemented by one-on-one meetings with the
program leadership for further guidance. Such virtual rotations also
allow for programs to evaluate students, their communication and
interpersonal skills and their overall fit with the residency program.
Facilitating an Applicant-Program Interaction for a
Successful MATCH
Previous studies have identified that resident satisfaction/
quality of life, operative volume, diversity of cases, and clinical
curriculum were the top-rated attributes which applicants seek while
www.annalsofsurgery.com | 109
Rajesh and Asaad
interviewing for GS residency programs.10,11 Although some of this
information is available on program web sites, interaction with
current residents offers direct first-hand information to applicants.
In this direction, GS can adopt a strategy similar to the recent
initiative taken by the American Council of Academic Plastic
Surgeons for plastic surgery residency applicants to MATCH
2021. The American Council of Academic Plastic Surgeons has
developed a platform for residency programs and applicants to
register for virtual ‘‘Meet-and-Greet’’ events.12 Through this model,
each participating program is offered a specified date and time,
where registered potential applicants interact with faculty and res-
idents through teleconferencing. These meets are organized with the
initial half of the 90-minute session being devoted to program
introduction, history, surgical volumes, and so forth presented by
the program leadership, followed by an opportunity for questions to
faculty. The latter half of these sessions comprise Q&A sessions with
residents alone for applicants to obtain a glimpse of the dynamics and
core values that each program fosters. As an extension, some plastic
surgery residency programs have started to host happy-hour sessions
where a small set of applicants interact directly with individual
residents in break-out rooms through videoconferencing. This allows
for more casual questions pertaining to everyday resident life,
expenses, and questions about the particular locality.
Ultimately, these forms of direct interaction will enable
students in applying selectively to their programs of interest, in
the absence of audition rotations.
Magnified Importance of Objective Criteria for
Evaluating Applicants
Virtual rotations, interviews, and preinterview dinners will
significantly impact applicant interaction with faculty and residents.
In this setting, the criteria for selection of interviewees is likely to
rely heavily on the paper application comprising test scores,
research, and awards among others.7 Several GS programs have
adopted personality tests to evaluate candidates before extending
interviews.13,14 Bell et al found that such testing was able to identify
certain applicant characteristics which were not apparent from other
application components, and that were positively associated with
success in residency.15 Further evidence discussing the validity of
personality testing in applicant selection comes from the article by
Shipper et al. The authors utilized the Computer-based Assessment
for Sampling Personal Characteristics test and found similar corre-
lations for the Computer-based Assessment for Sampling Personal
Characteristics and traditional faculty assessment of applicant
personality, in determining the applicant position on the final rank
order list.16 Large-scale implementation of such preinterview
assessments can be a useful objective adjunct for the upcoming
MATCH cycle.
Among other criteria, letters of recommendation (LORs) are
afforded a high degree of importance for surgery applicants.4,17
LORs from audition rotations are valued highly as the validity of
letters from the home institution is occasionally drawn into
question.7 However, the present narrative LOR is often subjective
and marred by inflation in terminology adopted to describe the
candidate.17 The existing situation limiting direct applicant eval-
uation can therefore serve as an impetus for GS to make the
paradigm shift towards standardized letters of recommendation.
Standardized letters of recommendation require that the letter
writer rank applicants in percentile for each evaluated skill
(clinical ability, research, communication/interpersonal skills,
medical knowledge etc) and detail the time-frame that he/she
has interacted with the applicant, thereby offering greater objec-
tivity in stratifying residency applicants.
110 | www.annalsofsurgery.com
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
Annals of Surgery  Volume 273, Number 1, January 2021
Skills Testing as a Plausible Supplement for MATCH
2021
The use of psychomotor skill and dexterity testing for selec-
tion of surgical trainees has been previous described with Gallagher
et al from Ireland discussing their selection process which included
the testing of technical skills and surgical aptitude for selecting
candidates for higher surgical training.18 Although skills testing has
previously been implemented for assessing surgical residents, there
is limited experience with the use of such methods for assessing GS
residency applicants in the United States.19,20 In a recently published
article, AlJamal et al described their experience of engaging medical
students in a simulation-based selection process, the performance on
which was utilized while formulating the rank list. Interviewees were
given adequate video-based instruction, and practice material was
mailed to enable practice before the skills testing, which ensued on
the interview day.21 The authors estimated a cost of less than $20
spent on this process per interviewee. In the era of virtual interviews,
such skills evaluations as assessed by requiring interviewees to
record their performance in suturing and other fundamental surgical
skills, can provide some insight into an applicant’s skill and dexterity.
These recordings can then be analyzed by faculty and senior resi-
dents to provide feedback during interviews and serve as an addi-
tional objective indicator of candidate proficiency. More importantly,
allowing applicants to practice before they submit a final video
recording would promote repetition and skill acquisition, and would
enable an evaluating faculty to gauge the commitment and invested
effort by the respective applicant.
Averting a Crisis – the Need for Limiting Interview
Number
Over the last several years, there has been a progressive
increase in the number of applications to GS residency programs.22
In 2019, matched US allopathic senior applicants applied to a median
of 51 programs compared to 68 programs for their nonmatching
counterparts. Although more competitive applicants receive more
interview invitations, time and cost limit the number of interviews
attended per candidate and this results in a fair distribution of
interviews to all applicants.22 The NRMP data shows that matched
US senior applicants received a median of 18 interviews and attended
a median of 14.22 However, with the advent of virtual interviews,
students are likely to attend several if not all interviews they receive.
Furthermore, the cost saved in terms of travel and lodging may
motivate students to apply more broadly to potentially increase their
chances at obtaining residency positions. This may culminate in a
great disparity in terms of interview number between different sets of
applicants because programs seeking the brightest applicants may
interview candidates who are likely to match higher up on their own
rank list. A plausible solution to this problem would be a recom-
mendation from the Association of American Medical Colleges,
limiting the number of interviews accepted per applicant per spe-
cialty. This would force applicants to accept interviews only at the
limited number of programs they would strongly consider for
future training.
In GS, the NRMP Charting Outcomes data demonstrates that
the mean number of continuous ranks (reflective of the number of
attended interviews) was 13 for matching senior applicants and 5 for
nonmatching applicants. The accompanying graph referenced
showed that the probability of matching was approximately 90%
with 11 interviews, 95% with 13 interviews and almost 100% with 17
interviews.23 Therefore, a limit of 15 to 17 interviews in GS per
applicant could offer a more level playing field and may reduce the
possibility of programs entering the Supplemental Offer and Accep-
tance Program (SOAP) to fill their residency positions. Along similar
ß 2020 Wolters Kluwer Health, Inc. All rights reserved.
Annals of Surgery  Volume 273, Number 1, January 2021
lines, the novel concept of an ‘‘interview match’’ before the actual
NRMP match was discussed by Melcher et al in JAMA while
analyzing cost and time spent on surgical fellowship interviews.
The authors described a potential option of ranking by both appli-
cants and programs for the available interview spots, which could
subsequently maximize the opportunity for interested candidates to
receive interviews from mutually interested programs.24 Although
these solutions would require careful thought and planning, they
constitute viable options to increase the likelihood of a successful
residency application cycle.
CONCLUSIONS
MATCH 2021 will be the first of its kind owing to the COVID-
19 pandemic. Limited away rotation opportunities significantly affect
the ability to gauge the applicant-program fit and virtual rotations
could prove useful in filling this void. ‘‘Meet-and-Greet’’ events
conducted before the application submission could offer insight into
the training environment at each residency program. Personality tests
before extending interviews, standardized LORs for all applicants and
skill assessments for interviewees may complement a program direc-
tor’s armamentarium while evaluating prospective candidates. Finally,
an interview limit or an ‘‘interview match’’ process before the actual
NRMP match, may serve as a potential solution to the applicant
disparity magnified by virtual interviews, and provide a fair playing
field for a broad range of GS residency applicants.
REFERENCES
1. Coronavirus (COVID-19) Resource Hub. AAMC. Available at: https://
www.aamc.org/coronavirus-covid-19-resource-hub. Accessed June 15, 2020
2. APDS Position Statement 2020–2021 Applicant Cycle. Association of Pro-
gram Directors in Surgery APDS Position Statement 2020–2021 Applicant
Cycle Comments. Available at: https://apds.org/2020/05/9294/. Accessed
June 15, 2020
3. Tseng J. How Has COVID-19 Affected the Costs of the Surgical Fellowship
Interview Process? J Surg Educ. 2020;77:999–1004. doi:10.1016/j.jsurg.
2020.05.018.
4. Association of American Medical Colleges, National Residency Match Pro-
gram Results of the 2018 NRMP Program Director Survey. Available at:
https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2018/07/
NRMP-2018-Program-Director-Survey-for-WWW.pdf. Accessed June 15,
2020
5. Higgins E, Newman L, Halligan K, et al. Do audition electives impact match
success? Med Educ Online. 2016;21:31325.
6. Khoushhal Z, Hussain MA, Greco E, et al. Prevalence and causes of attrition
among surgical residents. JAMA Surg. 2017;152:265–272.
7. Boyd CJ, Inglesby DC, Corey B, et al. Impact of COVID-19 on away rotations
in surgical fields. J Surg Res. 2020;255:96–98.
ß 2020 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
Evaluating the 2021 Surgery Applicants
8. Calhoun KE, Yale LA, Whipple ME, et al. The impact of COVID-19 on
medical student surgical education: Implementing extreme pandemic response
measures in a widely distributed surgical clerkship experience. Am J Surg.
2020;220:44–47.
9. Sullivan SA, Bingman E, O’rourke A, et al. Piloting Virtual Surgical Patient
Cases with 3rd-year medical students during the surgery rotation. Am J Surg.
2016;211:689–696.
10. Parker A, Petroze R, Schirmer B, et al. Surgical residency market research-
what are applicants looking for? J Surg Educ. 2013;70:232–236.
11. Stefanidis D, Miles W, Greene F. Factors influencing residency choice of
general surgery applicants—how important is the availability of a skills
curriculum? J Sur Edu. 2009;66:325–329.
12. Acaplasticsurgeons.org. 2020. American Council of Academic Plastic Sur-
geons - Meet & Greets. [online] Available at: https://acaplasticsurgeons.org/
events/Meet-and-Greets.cgi. Accessed July 14, 2020
13. Gardner AK, Cavanaugh KJ, Willis RE, et al. If you build it, will they come?
Candidate completion of preinterview screening assessments. J Surg Educ.
2019;76:1534–1538.
14. Hughes B, Perone J, Cummins C, et al. Personality testing may identify
applicants who will become successful in general surgery residency. J Sur Res.
2019;233:240–248.
15. Bell R, Fann S, Morrison J, et al. Determining personal talents and behavioral
styles of applicants to surgical training: a new look at an old problem, part II. J
Surg Educ. 2012;69:23–29.
16. Shipper E, Mazer L, Merrell S, et al. Pilot evaluation of the computer-based
assessment for sampling personal characteristics test. J Surg Res.
2017;215:211–218.
17. Rajesh A, Rivera M, Asaad M, et al. What are we really looking for in a letter
of recommendation? J Surg Educ. 2019;76:e118–e124.
18. Gallagher AG, Leonard G, Traynor OJ. Role and feasibility of psychomotor
and dexterity testing in selection for surgical training. ANZ J Surg.
2009;79:108–113.
19. Chandra A, Rajesh A, Backstrom C, et al. Training, testing and remediating:
the importance of simulation for surgical trainees. ANZ J Surg. 2019;89:1364–
1365.
20. Aljamal Y, Prabhakar N, Saleem H, et al. Surgical interns in 2018: Objective
assessment suggests they are better but still lack critical knowledge and skill.
Surgery. 2019;165:1093–1099.
21. Aljamal Y, Prabhakar N, Rivera M, et al. A simulation-based selection process
for trying to identify medical students who will become outstanding general
surgery residents. Am J Surg. 2019;220:76–82.
22. Association of American Medical Colleges, National Residency Match Pro-
gram Results of the 2019 NRMP Applicant Survey. Available at: https://
mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2019/06/Appli-
cant-Survey-Report-2019.pdf. Accessed June 15, 2020
23. Association of American Medical Colleges, National Residency Match Pro-
gram Charing Outcomes in the Match: U.S. Allopathic Seniors. Available at:
https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2019/10/
Charting-Outcomes-in-the-Match-2018_Seniors-1.pdf. Accessed June 15,
2020
24. Melcher M, Ashlagi I, Wapnir I. Matching for fellowship interviews. JAMA.
2018;320:1639–1640.
www.annalsofsurgery.com | 111