Clinical Review & Education

JAMA Surgery | Review

Entrustable Professional Activities in Surgery

A Review
Kelsey B. Montgomery, MD; John D. Mellinger, MD; Brenessa Lindeman, MD, MEHP

CME at [Link]
IMPORTANCE Entrustable professional activities (EPAs) compose a competency-based
education (CBE) assessment framework that has been increasingly adopted across medical
specialties as a workplace-based assessment tool. EPAs focus on directly observed behaviors
to determine the level of entrustment a trainee has for a given activity of that specialty. In
this narrative review, we highlight the rationale for EPAs in general surgery, describe current
evidence supporting their use, and outline some of the practical considerations for EPAs
among residency programs, faculty, and trainees.

OBSERVATIONS An expanding evidence base for EPAs in general surgery has provided
moderate validity evidence for their use as well as practical recommendations for
implementation across residency programs. Challenges to EPA use include garnering buy-in
from individual faculty and residents to complete EPA microassessments and engage in
timely, specific feedback after a case or clinical encounter. When successfully integrated into
a program’s workflow, EPAs can provide a more accurate picture of residents’ competence
for a fundamental surgical task or activity compared with other assessment methods.

CONCLUSIONS AND RELEVANCE EPAs represent the next significant shift in the evaluation
of general surgery residents as part of the overarching progression toward CBE among all Author Affiliations: Department of
US residency programs. While pragmatic challenges to the implementation of EPAs remain, Surgery, University of Alabama
the best practices from EPA and other CBE assessment literature summarized in this review at Birmingham (Montgomery,
Lindeman); American Board
may assist individuals and programs in implementing EPAs. As EPAs become more widely of Surgery, Philadelphia,
used in general surgery resident training, further analysis of barriers and facilitators to Pennsylvania (Mellinger).
successful and sustainable EPA implementation will be needed to continue to optimize and Corresponding Author: Brenessa
advance this new assessment framework. Lindeman, MD, MEHP, Department of
Surgery, University of Alabama
at Birmingham, 1808 7th Ave S,
JAMA Surg. doi:10.1001/jamasurg.2023.8107 Boshell Diabetes Building Ste 506,
Published online March 13, 2024. Birmingham, AL 35233 (blindeman@
[Link]).

E
ntrustable professional activities (EPAs) compose a com- grams; and future integration of EPAs in surgical subspecialty resi-
petency-based assessment framework focusing on di- dency and fellowship programs.
rectly observable tasks that together make up the core as-
pects of a given specialty. Following their original description by ten
Cate,1 EPAs have been increasingly adopted by multiple medical spe-
Rationale for EPAs in Surgery
cialties as well as undergraduate medical education. As part of the
broader movement toward competency-based education (CBE) in Competency-Based Education Model
general surgery training, the American Board of Surgery (ABS) de- Current requirements for graduating general surgery residents in-
veloped and recently published a set of 18 core EPAs for general sur- clude case volume– and time-based (ie, weeks of training) thresh-
gery as an assessment framework for trainees at all US residency olds, despite the variable number of cases or training level at which
programs.2 a resident may actually achieve competence to perform a given op-
Along with a national pilot study of EPAs in general surgery led eration or clinical task. Ongoing concerns around graduating resi-
by the ABS, multiple institutions have studied the validity and imple- dent preparedness have been expressed by both program direc-
mentation of EPAs as part of their assessment of surgical residents. tors and residents themselves, and residents have reported lower
While the incorporation of EPA assessments into the evaluation of self-efficacy compared with faculty perceptions of their abilities.3-8
surgical trainees is still relatively new, successes and challenges from These concerns have arisen within the context of seismic changes
these studies may provide important context as surgical residency in training, including the evolution of the practice of general sur-
programs across the United States integrate this assessment tool into gery, increasing regulatory requirements of attending physicians with
their current processes. concomitant decreasing resident autonomy, and changes in Ac-
In this review, we explore the rationale for EPAs in general sur- creditation Council for Graduate Medical Education (ACGME) duty-
gery; practical implications for trainees, faculty, and residency pro- hour regulations.9,10 Recognizing this disconnect in resident evalu-

[Link] (Reprinted) JAMA Surgery Published online March 13, 2024 E1

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Clinical Review & Education Review Entrustable Professional Activities in Surgery

Figure. Conceptual Framework of Levels of Competency-Based Education

subcompetencies
Subcompetency milestones: building blocks
Milestones

-1
MK “This resident demonstrates comprehensive knowledge of the varying
patterns of disease presentation and alternative and adjuvant treatments
-1
MK for patients with surgical conditions.”

Competency-based education
Competency

Competency domains: creating a strong foundation

domains

Medical knowledge Patient care Interpersonal communication

Professionalism Practice-based learning System-based practice

EPAs: putting it all together

EPAs

“I would trust this resident to treat a patient with gallbladder disease

with (eg, indirect supervision).”
EPAs indicates entrustable
professional activities.

ation and preparedness, surgical education leaders have driven a mean (SD) of 5.6 (13.4) microassessments but a median of only 1 mi-
transition over the last 2 decades toward CBE frameworks, which croassessment per resident.19 The pilot study allowed programs to
focus on more directly assessing resident competence instead of re- implement EPAs and collect microassessment data using methods
liance on surrogate measures such as case volumes.11,12 The CBE of their choosing, and this freedom, plus differences in program struc-
model for surgical education has drawn considerable interest from tures and cultures, likely contributed to the variability of EPA utili-
surgical educators; proponents highlight its emphasis on evaluat- zation during the pilot. Despite potential challenges for the wide-
ing residents’ abilities to perform relevant operative and nonoper- spread integration of the full suite of general surgery EPAs, overall
ative skills and de-emphasizing the checklist or “bean-counting” style results from this national EPA pilot study were encouraging and em-
of resident assessment that has been previously used.13-16 phasized the importance of ongoing educational efforts by the ABS
toward program faculty, trainees, and administrative staff. The full
Surgical EPA Development and Pilot Study list of core EPAs for general surgery as published by the ABS is pre-
Within the CBE educational model, EPAs represent an assessment sented in the Box.
framework that focuses on observable workplace-based micro-
assessments of the level of supervision needed for a trainee to per- Additional Validity Evidence for EPAs in Surgery
form a given task (Figure). As first described by ten Cate in 2005,1 A growing body of evidence from single- and multi-institution stud-
EPAs should consist of the essential activities of a medical specialty ies evaluating the use of EPAs for the evaluation of surgical resi-
and be directly observable. Incorporation of the EPA framework dents has developed over the last half decade as the discussion
into general surgery has been led by the ABS in collaboration with around incorporating EPAs nationally has evolved. Multiple studies
the ACGME Surgery Review Committee, American College of Sur- have presented data supporting the external validity of EPAs, in-
geons, and Association of Program Directors in Surgery. Together, cluding increasing entrustment levels with increasing resident post-
these surgical education leaders designed a pilot study for EPAs in graduate-year level and ability to achieve full entrustment (ie, un-
general surgery that was conducted from 2018 to 2020 at 28 sur- supervised practice).20-22 In 2 single-institution studies evaluating
gical residency programs, including both community and aca- all 5 pilot EPAs and the gallbladder EPA, respectively, study authors
demic centers.17,18 also demonstrated strong positive correlation between EPA micro-
This study evaluated 5 newly developed EPAs among a na- assessment ratings and ACGME Milestone scores.20,23 These cor-
tional sample of general surgery residents. Pilot EPAs included evalu- relations make intuitive sense given the deliberate mapping of
ation and management of a patient with right-lower-quadrant pain, Milestone competencies to EPAs by the ABS EPA development
inguinal hernia, gallbladder disease, and blunt or penetrating trauma, groups, while also providing evidence for their validity compared with
as well as performing general surgical consultation. As recently re- current forms of resident assessment.17
ported by Brasel et al,19 more than 6000 EPA microassessments In related work, Chen et al24 described the development and
were collected over the course of the pilot study, with 1763 sum- testing of 6 procedure-specific assessments they termed surgical
mative entrustment ratings (determined by program clinical com- EPAs based on the ABS Operative Performance Assessment frame-
petency committees at 6-month study intervals based on accumu- work. After testing these surgical EPAs (different from the ABS-
lated microassessment data) completed for 497 residents. Increasing developed set) with resident-faculty pairs plus a third surgeon ob-
levels of entrustment were seen with increasing postgraduate- server, they demonstrated strong positive correlation of resident
year level. entrustment ratings with the levels of guidance provided during the
Importantly, the authors noted wide variability in the numbers case, as well as their procedural performance and general skills. Pro-
of microassessments completed per resident by program, with a cedure-specific autonomy and entrustment, global operative com-

E2 JAMA Surgery Published online March 13, 2024 (Reprinted) [Link]

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Entrustable Professional Activities in Surgery Review Clinical Review & Education

uncaptured information about resident competency. Additionally,

Box. Core Entrustable Professional Activities (EPAs) EPA terminology and structure, including entrustment level rat-
of General Surgery ings among the 18 core EPAs, present a new lexicon for front-line
faculty and residents. Becoming fluent with this terminology will be
Three-Phase EPAsa facilitated by dedicated education to understand how the EPA as-
Right-lower-quadrant pain and appendicitis
sessment framework is different from end-of-rotation evaluations
Benign or malignant breast disease in both its philosophical approach and practical application.
Benign or malignant colon disease Despite these upfront implementation barriers, EPAs have sev-
Gallbladder disease eral potential advantages over current evaluation methods. First, in-
Inguinal hernia stead of having faculty try to estimate how well a resident demon-
Abdominal wall hernia strates isolated competencies in areas like Systems-Based Practice
or Professionalism, EPAs take advantage of the inherent, ad hoc com-
Acute abdomen
petency judgments that faculty are making every day while work-
Benign anorectal disease
ing with residents: that is, how much can I trust this resident to take
Small bowel obstruction
care of this patient? Prior work evaluating faculty decision-making
Thyroid and parathyroid disease around resident entrustment and autonomy supports that while spe-
Need for renal replacement therapy cific factors such as resident experience and clinical context can
Soft tissue infection (including necrotizing soft tissue infection) influence entrustment, this process happens empirically within the
Cutaneous and subcutaneous neoplasms surgical training environment.26-28 EPAs aim to serve as a natural ex-
Flexible gastrointestinal endoscopy tension of this process and quantify entrustment decision-making
by faculty through linking it to directly observed behaviors. By hav-
Perioperative care of the critically ill surgery patient (resuscitation,
procedures, postresuscitation) ing such a framework of expected behaviors at each level of entrust-
ment as a reference, faculty may be more consistent in determin-
Blunt or penetrating trauma (trauma bay, procedures, transition
of care) ing where residents fall in their entrustment level based on these
observed behaviors.
Two-Phase EPA (Evaluation and Management)
From a resident standpoint, EPAs offer an opportunity for re-
Severe acute or necrotizing pancreatitis
peated, real-time feedback based on directly observed perfor-
Single-Phase EPA (Consultation) mance, which they may not otherwise receive with current evalu-
Provide surgical consultation ation methods. Instead of a single end-of-rotation evaluation,
a
The 3 phases are preoperative, intraoperative, and postoperative except residents will steadily accumulate formative feedback across a spec-
for the instances noted. trum of core general surgery domains during a given rotation and
as they progress in their training. Additionally, with an increased
focus on brief, real-time feedback using the EPA microassessment
petencies, and resident learning efficacy were found to be most in- tool, faculty and residents alike may become more accustomed to
fluential on performance.24 Although this study focused only on incorporating a quick, constructive debrief into their postoperative
intraoperative performance and its methods have not been re- or postclinic routines to help residents improve on specific opera-
peated using the ABS general surgery EPAs, the overlap of opera- tive or clinical decision-making skills.29,30
tions included in the intraoperative care phase in some EPAs (eg, lapa- Ultimately, successful incorporation of the EPA assessment
roscopic cholecystectomy, inguinal and ventral hernia repairs) framework will require buy-in from both residents and faculty, who
suggest that similar underlying resident factors may be drivers of will share the additional tasks but also are motivated to receive
variation in resident entrustment in the set of ABS EPAs. and provide specific, constructive, timely feedback that EPAs will
help to make part of their regular workflow. Based on a single-
institution study focused on EPA implementation during the EPA
pilot, empowering residents to generate EPA microassessment re-
Practical Considerations for EPA Implementation
quests and optimizing the timing of these requests may help in-
Trainees and Faculty crease their completion rate.31 These findings were consistent with
The integration of EPAs among all US general surgery residency pro- operative performance rating (OPR) studies, which emphasized the
grams presents both potential benefits and pragmatic challenges for importance of immediate rating completion, noting a significant de-
trainees and faculty (Table). The more obvious barriers include the crease in feedback clarity and specificity at more than 3 days after
additional time and effort on top of already-taxed workloads of sur- observation.29,30 Further insights from EPA implementation across
gical teaching faculty, as well as additional burden on residents to different residency programs will be needed to help address any on-
discuss the need for EPA microassessments with faculty and send going resident concerns about how EPAs are being used, as de-
evaluation requests. Similar to issues discussed by Williams et al25 scribed by Gupta et al32 from a survey of resident perceptions of pi-
at the time of changes to ACGME Milestones that planned to ex- lot EPAs. Practical guidance for faculty about expected frequency
tend the length of evaluation forms, program directors will likely of EPA microassessments completed per resident or per time pe-
feel pressure to balance this new requirement of faculty with either riod will likely be dictated by program-specific expectations, but
decreased workload in other areas (eg, shortened end-of-rotation assessment data collected over these initial years of EPA implemen-
evaluations) and/or net benefit in providing useful and otherwise tation may help clarify best practices for the future.

[Link] (Reprinted) JAMA Surgery Published online March 13, 2024 E3

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Clinical Review & Education Review Entrustable Professional Activities in Surgery

Table. Summary of Benefits, Facilitating Factors, and Barriers for EPA Implementation

Benefits Facilitators Barriers

For trainees and faculty
Closer estimation to residents’ true Easy-to-use smartphone-based app Added time requesting and
competence than Milestones to request and complete EPA completing EPA microassessments
microassessments
Real-time feedback, ability to trend Interactive dashboards available to New terminology requiring
microassessment results over a single trainees and faculty with EPA data dedicated training to educate
rotation faculty raters and trainees
Encourage postoperative or postclinic Standardized rating scale validated
quick debrief in prior work
For the residency program
Increase in real-time assessment of Interactive dashboards available to Administrative tasks of organizing
residents to track progress and identify program directors to identify trends EPA microassessment collection
struggling trainees in resident performance and reminding faculty and trainees
Mapping EPAs to Milestones allows for Opportunities to reduce Identifying local champions among
more streamlined Milestone ratings for end-of-rotation burden by using faculty, trainees, and department Abbreviations: EPA, entrustable
program CCCs using EPA data EPA data to fill performance leadership professional activity; CCC, clinical
assessment need competency committee.

Residency Programs ity, prior work studying OPR assessments from Williams et al36 dem-
As with faculty and trainees, residency programs will also face their onstrated reliability and validity, noting specific trends in ratings such
own unique barriers and facilitators to EPA implementation. Based as increasing OPR scores with increasing resident postgraduate-
on work from Stahl et al,31 factors that supported completion of EPA year level. The authors noted that the use of 5 to 7 different faculty
microassessments during the pilot study included development of raters would reasonably control for rating idiosyncrasies between
a user-friendly mobile application for assessment data collection and judges, with this number of raters likely feasible for most residency
dedicated resident and faculty education around EPAs and levels of programs over the course of a 5-year training period.36 In related
entrustment. Prior work from George et al33 studying the duration work, they also noted that 2 to 3 OPRs per month provided a suffi-
of faculty training needed to achieve performance rating reliability cient amount of data to support decision-making around resident
for OPRs demonstrated that approximately 1 hour of training was progression.37 Further results from study of the SIMPL-OR applica-
sufficient, which may be a starting point for faculty training as an ini- tion suggested that 23 ratings are needed for faculty raters to achieve
tial orientation to the EPA framework. Others have also highlighted reproducible autonomy ratings for laparoscopic cholecystectomy,
the critical importance of local champions within the program, compared with 60 ratings for an undifferentiated mixture of gen-
including departmental leadership as well as faculty and resident eral surgery operations.38 This suggests that the learning curve
champions.31,34 Some institutions have used microassessment for faculty will vary based on the complexity and volume of proce-
completion targets as a strategy to increase EPA use, such as defin- dures they are rating. While anchoring to a specific number may nei-
ing a certain number of microassessments that each resident should ther be feasible nor necessary for the breadth of EPAs for general
receive per rotation or defining which of the EPAs should be evalu- surgery and their separate phases of care, having a target for resi-
ated on specific services. On a more practical level, simple interven- dents and faculty on a per-rotation or per–time period basis may
tions such as regularly scheduled reminder emails to residents help set a generalizable expectation for how to achieve a sufficient
and faculty helped prompt requests for EPA microassessments and range of EPA microassessments in working toward entrustment.
faculty completion.31 While incorporating EPAs will involve new types of tasks for resi-
The primary data collection tool that will be used for implemen- dency program leadership and administrative staff, the product of
tation and tracking of general surgery EPA data is the ABS-EPA smart- this labor should be a wealth of assessment data that will not only
phone-based application hosted by the tool System for Improving help benchmark residents’ progress over time in ways that end-of-
and Measuring Procedural Learning (SIMPL), which the ABS is pro- rotation evaluations are not able to provide, but hopefully also aid
viding to all residency programs at no cost. The SIMPL-OR platform in identifying struggling residents earlier than could be accom-
was originally designed for intraoperative performance assess- plished previously. For program clinical competency committees, EPA
ment and has been previously studied in multi-institutional work that data should provide a more consistent and repeated source of resi-
demonstrated high uptake by participating programs, including col- dent performance information and could also be used to help gen-
lection of more than 6000 3-question OPR assessments at 13 resi- erate required ACGME Milestone ratings.39 This is supported by both
dency programs during a 6-month data collection period.34 Initial the intentional mapping of EPAs to 5 to 7 subcompetencies and as-
analyses of this early study of the SIMPL platform described posi- sociated milestones during their development, along with data from
tive perceptions of the app’s ease of use and value of narrative feed- some confirmatory studies bolstering this correlation.17,18,20 As EPAs
back for residents, as well as difficulties reported in overcoming are increasingly adopted within a program, program leadership may
inertia of changing daily workflows to include these assessments be able to reduce the end-of-rotation evaluation burden to the mini-
and creating a culture of participation that encourages instead of mum information needed to supplement Milestone ratings, as the
requires SIMPL use.34,35 majority of residents’ performance assessments would have been
While the expected number of EPA microassessments needed captured through EPA microassessments.39
for a resident to achieve entrustment at the practice-ready level is Importantly, while EPAs will serve as an important data source
unknown and will likely vary based on context and case complex- to inform clinical competency committee ratings of Milestones, not

E4 JAMA Surgery Published online March 13, 2024 (Reprinted) [Link]

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Entrustable Professional Activities in Surgery Review Clinical Review & Education

all subcompetencies are represented among the general surgery ner oriented to competency end points. These are defined generi-
EPAs through EPA-Milestone mapping, as not all are directly observ- cally rather than by detailing discretely observed behaviors in a spe-
able in the context of a clinical encounter (ie, PROF-3, which as- cific clinical context; collectively, they are not able to define a
sesses completion of administrative tasks). Additionally, some sub- discipline, but rather describe a set of skills that individuals from
competencies are represented sparingly in the EPA set (eg, PROF-4 multiple disciplines (general surgery, gynecology, urology, etc) might
assessing self-care and PBLI-2 assessing personalized learning plans perform. Despite these differences, such approaches create align-
are only mapped to a single EPA), emphasizing the continued need ment around competency-directed training for educators and train-
for alternative methods of evaluating these subcompetencies and ees. This kind of thought coherence in educational planning to de-
their Milestones for required semiannual ratings. fined competence end points that harmonize with those captured
in the EPA clinical skills assessment framework may serve educa-
tors and trainees well in the conceptual shift from time- and volume-
founded training paradigms to competency-based ones.
Future Implications
Expansion of EPAs to Subspecialty Residency Potential for Bias
and Fellowship Programs Although EPAs are grounded in discrete, observable behaviors, which
EPAs are also in different stages of development and implementa- lend the view that they are based on more objective information,
tion for multiple surgical specialties, both subspecialty integrated they remain subjective assessments completed by an individual rater
residency programs and independent fellowship programs follow- and thus subject to bias. While studies of other workplace-based as-
ing general surgery training. With similar challenges in resident and sessments in surgery have demonstrated concern for bias on the ba-
fellow preparedness being discussed in the vascular surgery edu- sis of gender,43-45 early data from tools that use observable behav-
cation community,6 a move toward a CBE model has also been on- iors as part of their rating scale have not shown similar differences
going, including study of OPR assessments using the SIMPL-OR app on the basis of gender. Single-institution data from the EPA pilot trial
as well as EPA development through the ABS.40 Other EPA devel- in 2 studies demonstrated no differences in EPA scores from fac-
opment projects reported in the surgical education literature in- ulty based on resident gender,20,46 as did a multi-institution study
clude multiple subspecialties within the Fellowship Council for their of a different tool using third-party observer ratings of entrust-
accredited gastrointestinal surgery fellowships, including EPAs for ment behaviors in the operating room.47 These findings suggest that
abdominal wall, bariatric, foregut, and hepatobiliary surgery as well assessments linked to observable behaviors, such as EPAs, may not
as for flexible endoscopy.41 suffer the same types of bias as traditional subjective assessments.
Created in collaboration with relevant society groups, these Fel- No data are currently available related to race or ethnicity given the
lowship Council EPAs can be customized based on the subtype of gas- small sample size in the single-center studies, leaving open an im-
trointestinal surgery fellowship, with topics that move beyond core portant question that could be answered with a nationally repre-
general surgery residency EPAs to more advanced activities that a sentative dataset.
fellowship-trained surgeon would be expected to manage (eg, “Evalu-
ate and manage patients with parastomal hernias,” “Evaluate and Other Applications of Surgical EPAs
manage patients with esophageal motility disorders”).41 Other gen- Beyond their contribution to resident performance evaluations and
eral surgery subspecialty fellowship EPAs under development or progression through levels of training, another potential applica-
planned for future development include endocrine surgery (led by tion of general surgery EPA data includes exploring the relation-
the American Association of Endocrine Surgeons), complex general ship between graduating resident competency (as measured by ac-
surgical oncology (ABS), pediatric surgery (ABS), and trauma, burn, cumulated EPA microassessments) and patient outcomes. A similar
and surgical critical care (ABS) fellowships. Development and imple- approach using non–fellowship-trained general surgery residency
mentation of EPAs at the fellowship level presents an opportunity graduates’ ACGME Milestones ratings and risk-adjusted complica-
to better integrate trainee assessment across the continuum of sur- tion rates did not demonstrate a statistically significant relation-
gical training, identifying areas of overlap between general and sub- ship between Milestone ratings and early-practice surgeons’
specialty practice while highlighting the skill sets unique to subspe- outcomes.48 As acknowledged by the authors and in an accompa-
cialists. The opportunity to bring assessments from all of these nying commentary, Milestone ratings are likely an imperfect mea-
specialties together in a unified platform would help support fac- sure of resident competency at best, given their vague descriptors
ulty across training programs to assess all learners with whom they and heavily retrospective nature.23,48,49 EPAs may provide more spe-
work and track their progress over different stages of their training. cific data for understanding resident competency for a given pro-
cedure, and linking these data to large outcomes databases, ideally
Other CBE Workplace-Based Assessments at a national level, may provide a more fruitful evaluation of the re-
While EPAs represent a competency-oriented clinical skills assess- lationship between surgeon competence and patient outcomes, one
ment framework and are therefore foundational to CBE initiatives of the ultimate goals of surgical education research.
in surgical training, there is more to CBE reform than EPAs. A full-
orbed approach to CBE in medicine includes competency-directed
curricular reform and the use of simulation to accelerate skill devel-
Conclusions
opment outside clinical contexts. An example of the latter is the Fun-
damentals of Laparoscopic Surgery program.42 Recent updates have EPAs represent a significant shift in the evaluation of general sur-
incorporated an EPA-like model to ground the revisions in a man- gery residents as part of the progression toward CBE and prioritiza-

[Link] (Reprinted) JAMA Surgery Published online March 13, 2024 E5

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Clinical Review & Education Review Entrustable Professional Activities in Surgery

tion of assessing resident preparedness for independent practice. ment EPAs. As EPAs become more widely used in general surgery
While pragmatic challenges to the implementation of EPAs remain, residency programs, further analysis of barriers and facilitators to suc-
best practices from EPA and other CBE assessment literature sum- cessful and sustainable EPA implementation will be needed to
marized in this review may assist individuals and programs to imple- continue to optimize this new assessment framework.

ARTICLE INFORMATION 10. Klingensmith ME, Lewis FR. General surgery 23. Albright JB, Meier AH, Ruangvoravat L,
Accepted for Publication: October 23, 2023. residency training issues. Adv Surg. 2013;47:251-270. VanderMeer TJ. Association between entrustable
doi:10.1016/[Link].2013.05.001 professional activities and Milestones evaluations:
Published Online: March 13, 2024. real-time assessments correlate with semiannual
doi:10.1001/jamasurg.2023.8107 11. Williams RG, George BC, Bohnen JD, et al.
A proposed blueprint for operative performance reviews. J Surg Educ. 2020;77(6):e220-e228.
Conflict of Interest Disclosures: Dr Montgomery training, assessment, and certification. Ann Surg. doi:10.1016/[Link].2020.07.027
reported being a resident member of the EPA 2021;273(4):701-708. doi:10.1097/SLA. 24. Chen XP, Harzman A, Cochran A, Ellison EC.
Writing Group and receiving grant support from 0000000000004467 Evaluation of an instrument to assess resident
the Agency for Healthcare Research and Quality surgical entrustable professional activities (SEPAs).
(T32 HS013852) outside the submitted work. 12. Lindeman B, Minter RM. Creating a shared
mental model for EPAs in surgery. Am J Surg. 2020; Am J Surg. 2020;220(1):4-7. doi:10.1016/[Link].
Dr Mellinger reported being the current vice 2019.08.026
president of the American Board of Surgery. 220(1):2-3. doi:10.1016/[Link].2020.04.032
Dr Lindeman reported being the current councilor 13. ten Cate O, Scheele F. Competency-based 25. Williams RG, Dunnington GL, Mellinger JD,
of the American Board of Surgery, director of the postgraduate training: can we bridge the gap Klamen DL. Placing constraints on the use of
General Surgery Board, and facilitator of the EPA between theory and clinical practice? Acad Med. the ACGME milestones: a commentary on the
Writing Group. No other disclosures were reported. 2007;82(6):542-547. doi:10.1097/ACM. limitations of global performance ratings. Acad Med.
0b013e31805559c7 2015;90(4):404-407. doi:10.1097/ACM.
REFERENCES 0000000000000507
14. Greenberg JA, Minter RM. Entrustable
1. ten Cate O. Entrustability of professional professional activities: the future of 26. Chen XP, Sullivan AM, Smink DS, et al. Resident
activities and competency-based training. Med Educ. competency-based education in surgery may autonomy in the operating room: how faculty
2005;39(12):1176-1177. doi:10.1111/j.1365-2929.2005. already be here. Ann Surg. 2019;269(3):407-408. assess real-time entrustability. Ann Surg. 2019;269
02341.x doi:10.1097/SLA.0000000000003153 (6):1080-1086. doi:10.1097/SLA.
0000000000002717
2. Entrustable professional activities. American 15. Stahl CC, Minter RM. New models of surgical
Board of Surgery. Accessed August 29, 2023. training. Adv Surg. 2020;54:285-299. doi:10.1016/j. 27. Sandhu G, Thompson-Burdine J, Nikolian VC,
[Link] yasu.2020.05.006 et al. Association of faculty entrustment with
resident autonomy in the operating room. JAMA Surg.
3. Meyerson SL, Teitelbaum EN, George BC, 16. Lindeman B, Sarosi GA. Competency-based 2018;153(6):518-524. doi:10.1001/jamasurg.2017.6117
Schuller MC, DaRosa DA, Fryer JP. Defining the resident education: the United States perspective.
autonomy gap: when expectations do not meet Surgery. 2020;167(5):777-781. doi:10.1016/[Link]. 28. Nieboer P, Huiskes M, Cnossen F, Stevens M,
reality in the operating room. J Surg Educ. 2014;71 2019.05.059 Bulstra SK, Jaarsma DADC. The supervisor’s toolkit:
(6):e64-e72. doi:10.1016/[Link].2014.05.002 strategies of supervisors to entrust and regulate
17. Brasel KJ, Klingensmith ME, Englander R, et al. autonomy of residents in the operating room.
4. Anderson TN, Payne DH, Dent DL, Kearse LE, Entrustable professional activities in general Ann Surg. 2022;275(1):e264-e270. doi:10.1097/
Schmiederer IS, Korndorffer JR. Defining the deficit surgery: development and implementation. J Surg SLA.0000000000003887
in US surgical training: the trainee’s perspective. Educ. 2019;76(5):1174-1186. doi:10.1016/[Link].
J Am Coll Surg. 2021;232(4):623-627. doi:10.1016/j. 2019.04.003 29. Williams RG, Chen XP, Sanfey H, Markwell SJ,
jamcollsurg.2020.11.029 Mellinger JD, Dunnington GL. The measured effect
18. Lindeman B, Brasel K, Minter RM, Buyske J, of delay in completing operative performance
5. George BC, Bohnen JD, Williams RG, et al; Grambau M, Sarosi G. A phased approach: the ratings on clarity and detail of ratings assigned.
Procedural Learning and Safety Collaborative general surgery experience adopting entrustable J Surg Educ. 2014;71(6):e132-e138. doi:10.1016/j.
(PLSC). Readiness of US general surgery residents professional activities in the United States. Acad Med. jsurg.2014.06.015
for independent practice. Ann Surg. 2017;266(4): 2021;96(7S):S9-S13. doi:10.1097/ACM.
582-594. doi:10.1097/SLA.0000000000002414 0000000000004107 30. Williams RG, Kim MJ, Dunnington GL. Practice
guidelines for operative performance assessments.
6. Smith BK, Rectenwald J, Yudkowsky R, 19. Brasel KJ, Lindeman B, Jones A, et al. Ann Surg. 2016;264(6):934-948. doi:10.1097/SLA.
Hirshfield LE. A framework for understanding the Implementation of entrustable professional 0000000000001685
association between training paradigm and trainee activities in general surgery: results of a national
preparedness for independent surgical practice. pilot study. Ann Surg. 2023;278(4):578-586. 31. Stahl CC, Collins E, Jung SA, et al.
JAMA Surg. 2021;156(6):535-540. doi:10.1001/ doi:10.1097/SLA.0000000000005991 Implementation of entrustable professional
jamasurg.2021.0031 activities into a general surgery residency. J Surg Educ.
20. Brazelle M, Zmijewski P, McLeod C, Corey B, 2020;77(4):739-748. doi:10.1016/[Link].2020.
7. Mattar SG, Alseidi AA, Jones DB, et al. General Porterfield JR Jr, Lindeman B. Concurrent validity 01.012
surgery residency inadequately prepares trainees evidence for entrustable professional activities in
for fellowship: results of a survey of fellowship general surgery residents. J Am Coll Surg. 2022;234 32. Gupta A, Watkins AC, Fahey TJ, Barie PS,
program directors. Ann Surg. 2013;258(3):440-449. (5):938-946. doi:10.1097/XCS. Narayan M. Entrustable professional activities:
doi:10.1097/SLA.0b013e3182a191ca 0000000000000168 do general surgery residents trust them? J Surg Educ.
2020;77(3):520-526. doi:10.1016/[Link].2019.12.005
8. Jensen RM, Kearse LE, Anand A, Dent DL, 21. Kearse LE, Schmiederer IS, Anderson TN,
Korndorffer JR. The program director perspective: Dent DL, Payne DH, Korndorffer JR Jr. American 33. George BC, Teitelbaum EN, Darosa DA, et al.
perceptions of PGY5 residents’ operative Board of Surgery entrustable professional activities Duration of faculty training needed to ensure
self-efficacy and entrustment. Am Surg. 2023;89 (EPAs): assessing graduating residents’ perception reliable or performance ratings. J Surg Educ. 2013;
(7):3098-3103. doi:10.1177/00031348231157862 of preoperative entrustment. J Surg Educ. 2021;78 70(6):703-708. doi:10.1016/[Link].2013.06.015

9. Oliver JB, McFarlane JL, Kunac A, Anjaria DJ. (6):e183-e188. doi:10.1016/[Link].2021.09.004 34. Bohnen JD, George BC, Williams RG, et al;
Declining resident surgical autonomy and 22. Steiman J, Sullivan SA, Scarborough J, et al. Procedural Learning and Safety Collaborative
improving surgical outcomes: correlation does Measuring competence in surgical training through (PLSC). The feasibility of real-time intraoperative
not equal causality. J Surg Educ. 2023;80(3): assessment of surgical entrustable professional performance assessment with SIMPL (system for
434-441. doi:10.1016/[Link].2022.10.009 activities. J Surg Educ. 2018;75(6):1452-1462. improving and measuring procedural learning):
doi:10.1016/[Link].2018.05.004 early experience from a multi-institutional trial.
J Surg Educ. 2016;73(6):e118-e130. doi:10.1016/j.
jsurg.2016.08.010

E6 JAMA Surgery Published online March 13, 2024 (Reprinted) [Link]

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024
 Entrustable Professional Activities in Surgery Review Clinical Review & Education

35. Eaton M, Scully R, Schuller M, et al. Value and 40. Cox ML, Weaver ML, Johnson C, et al. Ann Surg Open. 2023;4(1):e256. doi:10.1097/AS9.
barriers to use of the SIMPL tool for resident Early findings and strategies for successful 0000000000000256
feedback. J Surg Educ. 2019;76(3):620-627. implementation of SIMPL workplace-based 45. Chen JX, Chang EH, Deng F, et al. Autonomy
doi:10.1016/[Link].2019.01.012 assessments within vascular surgery residency in the operating room: a multicenter study of
36. Williams RG, Sanfey H, Chen XP, Dunnington GL. and fellowship programs. J Vasc Surg. 2023;78(3): gender disparities during surgical training. J Grad
A controlled study to determine measurement 806-814.e2. doi:10.1016/[Link].2023.04.039 Med Educ. 2021;13(5):666-672. doi:10.4300/JGME-
conditions necessary for a reliable and valid 41. Hanson MN, Pryor AD, Jeyarajah DR, et al; D-21-00217.1
operative performance assessment: a controlled FC EPA Pilot Working Group. Implementation of 46. Padilla EP, Stahl CC, Jung SA, et al. Gender
prospective observational study. Ann Surg. 2012; entrustable professional activities into fellowship differences in entrustable professional activity
256(1):177-187. doi:10.1097/SLA.0b013e31825b6de4 council accredited programs: a pilot project. Surg evaluations of general surgery residents. Ann Surg.
37. Williams RG, Verhulst S, Colliver JA, Sanfey H, Endosc. 2023;37(4):3191-3200. doi:10.1007/s00464- 2022;275(2):222-229. doi:10.1097/SLA.
Chen X, Dunnington GL. A template for reliable 022-09502-5 0000000000004905
assessment of resident operative performance: 42. Gomez-Garibello C, Wagner M, Seymour N, 47. Thompson-Burdine J, Sutzko DC, Nikolian VC,
assessment intervals, numbers of cases and raters. Okrainec A, Vassiliou M. The entrustable et al. Impact of a resident’s sex on intraoperative
Surgery. 2012;152(4):517-524. doi:10.1016/[Link].2012. professional activities of laparoscopic surgery: entrustment of surgery trainees. Surgery. 2018;164
07.004 moving toward an integrated training model. Surg (3):583-588. doi:10.1016/[Link].2018.05.014
38. Williams RG, Swanson DB, Fryer JP, et al. Endosc. 2023;37(7):5335-5339. doi:10.1007/
s00464-023-10022-z 48. Kendrick DE, Thelen AE, Chen X, et al.
How many observations are needed to assess Association of surgical resident competency ratings
a surgical trainee’s state of operative competency? 43. Meyerson SL, Sternbach JM, with patient outcomes. Acad Med. 2023;98(7):
Ann Surg. 2019;269(2):377-382. doi:10.1097/SLA. Zwischenberger JB, Bender EM. The effect of 813-820. doi:10.1097/ACM.0000000000005157
0000000000002554 gender on resident autonomy in the operating
room. J Surg Educ. 2017;74(6):e111-e118. 49. Montgomery KB, Lindeman B. Using
39. Krecko LK, Jung S, Martin S, et al. Enhancing graduating surgical resident milestone ratings
the value of surgical entrustable professional doi:10.1016/[Link].2017.06.014
to predict patient outcomes: a blunt instrument
activities through integrative learning analytics. 44. Filiberto AC, Abbott KL, Shickel B, et al. for a complex problem. Acad Med. 2023;98(7):
J Surg Educ. 2023;80(10):1370-1377. doi:10.1016/j. Resident operative autonomy and attending verbal 765-768. doi:10.1097/ACM.0000000000005165
jsurg.2023.07.018 feedback differ by resident and attending gender.

[Link] (Reprinted) JAMA Surgery Published online March 13, 2024 E7

© 2024 American Medical Association. All rights reserved.

Downloaded from [Link] by Universidad Del Rosario user on 03/14/2024