EDUCATIONAL INNOVATION

Using Kotter’s Change Management Framework to

Redesign Departmental GME Recruitment
Matthew C. Miles , MD, MEd Christopher Kelly , MD
Karl M. Richardson , MD Jacqueline Lippert, MD
Rachel Wolfe , MD Nina Mastandrea, BS
Kristen Hairston , MD, MPH Zachary Pruitt , MD
Maryjo Cleveland, MD

ABSTRACT
Background In light of the COVID-19 pandemic, dramatic change in the graduate medical education (GME) trainee recruitment
process was required. Kotter’s 8-Step Change Model is a change management framework that has been successfully applied to a
variety of GME initiatives but not for recruitment redesign.
Objective To implement major change in program recruitment during the COVID-19 pandemic while maintaining Match
outcomes and a high-quality candidate experience.
Methods In 2020, we applied Kotter’s 8 steps to implement major changes to program recruitment for a department of internal
medicine including 15 GME programs (1 internal medicine residency and 14 subspecialty fellowships). We collected each program’s
Match fill rates and used Google Analytics to collect monthly website traffic for the year prior to our change process and the subsequent
2 years. Standardized post-interview survey questions were created, and these results were reviewed for descriptive analysis.
Results We successfully used Kotter’s 8 steps to change recruitment to a virtual format. Program fill rates remained high after
implementation. Website engagement improved with peak monthly page rates doubling over previous values. During the highest
traffic month, the average time on site increased for 7 programs, while the bounce rate decreased by more than half for 10 programs.
Candidate descriptive feedback was positive.
Conclusions The application of Kotter’s 8 steps guided major changes to GME recruitment for 15 programs and was associated with
maintained Match fill rates and increased website engagement.

Introduction not aware of previous descriptions applying it to

recruitment. The objective of our article is to describe
As our graduate medical education (GME) programs
the application of Kotter’s framework in our redesign
approached the recruitment season of 2020, we
efforts, describe our interventions, and report on the
recognized a need to change our practices drastically
outcomes, including feasibility and acceptability of
and rapidly due to the COVID-19 pandemic. We
using the framework to guide change and the
immediately redesigned recruitment for a virtual
interventions themselves.
environment where candidates could not make in-
person visits. Although expert guidance was later
provided for this specific situation (including methods
Methods
of increasing programs’ digital visibility,1 transition- Setting and Participants
ing to virtual interviews,2 and selecting social media This study was completed at Wake Forest University
platforms3), these insights were not available at the School of Medicine, within the Department of
time of our initial planning. We thus implemented a Internal Medicine, which includes 15 GME programs
well-known change management strategy developed (one general internal medicine residency and a wide
by John P. Kotter and known as Kotter’s 8-Step range of subspecialty fellowships). In 2020, 2 faculty
Change Model,4,5 which has been successfully applied co-chairs assembled an 8-person virtual recruitment
to many industries, including health care.6,7 GME committee at the direction of the chair of medicine. A
programs have used this model to address the departmentally funded project manager was assigned
transition to competency-based medical education,8 to assist with administrative tasks including commu-
didactics redesign,9 and trainee wellness,10 but we are nication and group organization.

DOI: http://dx.doi.org/10.4300/JGME-D-22-00191.1 Interventions

Editor’s Note: The online version of this article contains the mind
map exercises used in the study, resources provided, a summary of We followed Kotter’s 8 steps for our process using
implemented major changes, and further data from the study. committee video conference meetings every 2 weeks

98 Journal of Graduate Medical Education, February 2023

 EDUCATIONAL INNOVATION

for 8 sessions and smaller subcommittee meetings in Objectives

the off weeks. Each meeting led to tangible and Apply Kotter’s change management framework to graduate
deadline-driven action items that were detailed in medical education (GME) recruitment redesign for multiple
emailed meeting minutes. At the same time, we programs within a department of internal medicine.
alerted programs via email of both the need for Findings
Change from in-person to virtual recruitment strategies was
change and impending support that our group would managed with maintained program fill rates and increased
provide in the coming months. levels of candidate website engagement.
Early discussions focused on 2 ways of process Limitations
thinking that ultimately informed larger group This study applied change management in the context of
brainstorming exercises: the candidate-centered strat- urgent recruitment changes for a single department brought
on by the COVID-19 pandemic. The success of this approach
egy and the resource-centered strategy. Mind map in other contexts is not addressed.
exercises were used to visualize early brainstorming
Bottom Line
considerations for these 2 strategies. Areas for specific Kotter’s 8 steps was utilized to successfully coordinate a
content development or improvement were distribut- large and complex change management project in GME
ed to all 15 programs. recruitment.
Below we describe steps 1 to 3 in Kotter’s 8 steps
for our process, which culminated in forming a Data on candidate perceptions of our recruitment
strategic vision and initiatives. Steps 4 to 8 of our process were limited to informal communications and
process follow from this implementation. All 8 steps some program-initiated surveys. Our task force
are outlined in TABLE 1. developed several standardized questions to be
included in post-interview surveys of candidates, but
Outcomes Measured not all programs used these questions. Due to this and
an overall low response rate, statistical analysis was
Our primary goals were maintaining program fill
not possible. Thus, summary description was per-
rates and providing a high-quality candidate experi- formed by the authors.
ence. Outcomes of the National Resident Matching This project was deemed exempt from Institutional
Program (NRMP) Medical Specialties Match were Review Board oversight by the authors according to
collected from the NRMP report ‘‘NRMP Program institutional policy.
Results 2018-2022 Specialties Matching Service.’’11
We considered also obtaining and reporting each Results
program’s maximum rank to fill, though program
directors did not agree to disclose this information. The application of Kotter’s 8 steps and resultant
To assess the candidate experience and acceptabil- interventions are described in TABLE 1. Here we detail
ity, we collected data on website engagement and steps 1 to 3, which resulted in specific actions to
candidate perceptions. Within Kotter’s framework, implement change.
these align with identifying short-term wins to help
Step 1: Create a Sense of Urgency
promote ongoing change.
Google Analytics was used to compare data from In March 2020, many societies including the Associ-
all internal medicine subspecialty fellowship program ation of American Medical Colleges, the Alliance for
websites from 3 recruitment seasons: 2019-2020, Academic Internal Medicine, and the NRMP recom-
2020-2021, and 2021-2022 (through December mended all programs commit to online interviews and
12
2021). We examined monthly website page views, virtual visits. The sense of urgency thus created was
mean time on site (measured in seconds), and bounce palpable, leading our department chair to select 2
rate (the percentage of visitors who left the web page faculty co-chairs to build and lead a team to redesign
or closed their browsers instead of clicking something GME recruitment strategy for all 15 programs within
on the page). Data were compared from pre- our department.
pandemic (2019-2020) and pandemic (2020-2021
and 2021-2022) recruitment seasons. We analyzed Step 2: Build a Guiding Coalition
monthly views for the main residency, separate from The co-chairs assembled a team of 6 additional
the fellowships, since the Match timings are separate. education-focused faculty members representing var-
September of each year had consistently high amounts ied educational roles, training programs, experience
of website traffic, and we specifically examined time with social media and information technology, and
on site and bounce rate for each program during this seniority. The group included 5 fellowship program
month. directors, one residency associate program director,

Journal of Graduate Medical Education, February 2023 99

EDUCATIONAL INNOVATION

TABLE 1
Virtual Recruitment Task Force Process According to Kotter’s 8 Steps of Change Management
Kotter’s Step Our Virtual Recruitment Process Approach
1. Create a sense of urgency &
Reviewed AAMC, AAIM, NRMP, and other organization recommendations that all
recruitment should be virtual for the 2020-2021 recruiting season.
&
Working with department chair and residency program director, workgroup co-chairs
reviewed and refined a draft charge and timeline for the workgroup.

2. Build a guiding coalition &

Co-chairs recommended additional GME faculty for the workgroup, selecting a diverse
representation of specialties, experience, and demography.

3. Form a strategic vision &

Held initial broad-scope discussions regarding the workgroup charge and areas of focus.
&
Used mind-mapping technique for brainstorming with time for reflection and revision.
&
Selected 5 focus areas and subcommittee working groups.
&
Identified ‘‘personal touch’’ as a guiding concept for implementation.

4. Enlist a volunteer army &

Communicated to all department training programs regarding the project, including
opportunity for dialogue and FAQs.
&
Solicited input and direct support from trainees and faculty for website and social media
additions in addition to video roles.

5. Enable action by removing &

Centralized contract with videographer and producer, including filming location.
barriers &
Developed suggested video content templates.
&
Purchased departmental video interviewing solution with training for coordinators.
&
Scaffolding support for website design, social media content generation, and interview
day structure were implemented.
* Specific website developers assigned to the project.

* Marketing department contacts and hiring of a new marketing and social media

content coordinator.
&
Interview gift developed, purchased, and postage funded through department.

6. Generate short-term wins &

Rapidly reviewed and discussed new website design and initial video products.
&
Rapidly developed, deployed, and then analyzed a social media survey exploring how our
current trainees learned about their respective program.
&
Launched program-specific landing page designed for applicants offered interviews.

7. Sustain acceleration &

Reviewed website completion at monthly department fellowship meetings.
&
Updated on all focus areas via email biweekly.
&
Updated with brief video on gift box development.
&
Just-in-time support given for interview day through program coordinators.
&
Post-interview, updated web page that targeted the ranking decision process and follow-
up email reminding applicants of key program features.
&
Post-Match, sent accordion postcards to celebrate new house staff and increase social
media exposure.

8. Institute change &

Ongoing planning for future recruitment years, including further integration of website
and social media content including candidate interactivity.
&
Measure and report on success of the class recruited during this process.

Abbreviations: AAMC, Association of American Medical Colleges; AAIM, Alliance for Academic Internal Medicine; NRMP, National Resident Matching
Program; GME, graduate medical education.

and 2 teaching faculty in general medicine. We met proposals to the department chair for funding of
for 1 hour, biweekly, via video conferencing for a total shared resources.
of 8 meetings (8 hours) over 4 months. Pairs of
members met more frequently as they took the lead Step 3: Form a Strategic Vision and Develop
on focus areas of our strategy. Time spent on work Initiatives
outside of committee meetings, including on email,
was not recorded. The faculty did not receive extra The vision for our change process was to implement a
funding support, but all held roles in leadership of recruitment strategy that would maintain a high
fellowship and residency programs and would have program fill rate and a high-quality candidate
been developing these types of activities individually experience. Specific initiatives were developed over
for them. The committee was allowed to present the course of our first 2 meetings, which were

100 Journal of Graduate Medical Education, February 2023

 EDUCATIONAL INNOVATION

FIGURE 1
September Page Views per Program
a
Program 11’s data is reported as one tenth of actual views since this program’s traffic far exceeded the others. All other programs’ data are actual page
views.

intentionally open-ended and utilized brainstorming. Representative comments include: ‘‘The gift box was
We elaborated on 2 themes: the candidate’s needs memorable as it showed effort from the program to
regarding recruitment and the program’s needs for give us a chance to learn about [the city],’’ ‘‘The
successful recruitment. Mind map exercises were used website is so far ahead of all other programs I
to visualize shared perspectives for these 2 strategies interview at,’’ and ‘‘I thought it was very well
(see online supplementary data). To create these organized. I was able to talk with the fellows
maps, we used the free version of MindMeister. In independently of faculty members before talking with
the candidate-centered map, we attempted to under- faculty members during my interview. I did appreciate
stand how the candidate engages with a potential the dedication to the website. The amount and quality
training program throughout the recruitment season. of the videos on the website as well as the information
We used this initial mind map to inform 5 key focus provided on the website was very helpful.’’
areas, which were each assigned to a pair of Final fill rates for our programs in pre-pandemic
committee members. The 5 focus areas were program (2020) and pandemic years are displayed in TABLE 2.
website, video content, social media content, inter- Match results showed 100% fill rates for all but 2
view day experience, and interview gift. Resources programs, including some programs in specialties
provided and major changes implemented are sum- with much lower national fill rate averages.
marized in the online supplementary data.
Discussion
Candidate Experience
Challenged with urgency, we were able to effectively
Number of monthly page views for our subspecialty facilitate large-scale changes to our recruitment
websites increased during the first year of the process for 15 programs using Kotter’s 8 steps. This
pandemic, from a previous peak of 7000 per month change management process allowed for coordination
to a new peak of 15 000 per month. These remained among multiple stakeholders and was associated with
similar during the second year. During both pandemic positive outcomes in website engagement and Match
years, peak views occurred in September for fellow- results.
ships and September or October for the core residency Implementing this process required time investment
(online supplementary data). Comparing September from several faculty and staff. These team members
page views year-over-year, there was a large increase were usual participants in program recruitment
as displayed in FIGURE 1. When comparing September activities, and some of the time spent on this change
time on site and bounce rate across 3 years, 7 process was in service of their usual job duties
programs saw an increase in average time on site, (contributing to feasibility). We did not specifically
and 10 programs saw bounce rates decrease by half track the additional number of hours spent on
(FIGURES 2 and 3). recruitment activities during our intervention year.
Two of our programs shared post-interview follow- Step 1 in Kotter’s process (Create a sense of urgency)
up survey data from 30 of 100 surveyed candidates. was particularly accelerated by the unique setting of

Journal of Graduate Medical Education, February 2023 101

EDUCATIONAL INNOVATION

FIGURE 2
September Bounce Rate per Program

FIGURE 3
September Time on Site per Program

TABLE 2
National and Local Positions Filled in the Medical Specialties Match (2020-2022)
National Local National Local National Local
a Positions Positions Positions Positions Positions Positions
Program
Filled Filled Filled Filled Filled Filled
2020, % 2020, % 2021, % 2021, % 2022, % 2022, %
1 93.1 100 92.8 100 98.3 100
4 76.6 50 85.1 50 85.3 100
5 54 100 53.6 25 53.6 50
6 99.7 100 99.7 100 99.6 100
7 97.7 100 99.3 100 98.0 100
8 79.3 100 87.7 100 82.1 100
9 99.8 100 99.7 100 99.8 100
10 99.6 100 99.2 100 99.7 100
12 99.7 100 100 100 99.1 80
14 97.2 100 97.3 100 97.8 100
15 60.1 100 71.2 100 69.2 100
a
Numbering is not sequential because only programs participating in the National Resident Matching Program are represented.

102 Journal of Graduate Medical Education, February 2023

 EDUCATIONAL INNOVATION

the COVID-19 pandemic and a shared sense of 4. Kotter JP. Leading change: why transformation efforts
urgency and purpose among our faculty. fail. Harvard Business Review. Published May-June
Other work in GME is relevant to our study. 1995. Accessed November 11, 2022. https://hbr.org/
Successful application of Kotter’s 8 steps has been 1995/05/leading-change-why-transformation-efforts-
shown for multiple areas of GME and now, in our fail-2
study, to the area of recruitment redesign. Others 5. Kotter JP. Leading Change. Harvard Business Review
have shown that bounce rate and web analytics are Press; 2012.
useful in evaluating improvement after a website 6. Stoller JK. Change: leadership essentials for chest
redesign,13 and our findings were similar. Lastly, ours medicine professional. Chest. 2021;159(4):1559-1566.
is the first study to report NRMP Match results doi:10.1016/j.chest.2020.09.094
during a time of recruitment redesign. 7. Stoller JK, Haney D, Burkhart J, et al. Physician-
Our study of these processes has limitations. We do ordered respiratory care vs physician-ordered use of a
not assert that our recruitment changes alone drove respiratory therapy consult service: early experience at
these favorable outcomes. Data on website engage- The Cleveland Clinic Foundation. Respir Care.
ment are influenced by many factors, and the societal 1993;38(11):1143-1154.
shift toward online interaction also contributed to 8. Dagnone JD, McEwen L, Taylor D, et al. Seven ways to
changes in engagement. Future data are needed to get a grip on implementing competency-based medical
confirm permanence. Data on Match rates are education at the program level. Can Med Educ J.
influenced by multiple factors, and true success of 2020;11(5):e92-e96. doi:10.36834/cmej.68221
recruitment will not be known until newly matched 9. Haas MRC, Munzer BW, Santen SA, et al.
trainees complete their training. Lastly, we could not #DidacticsRevolution: applying Kotter’s 8-step change
implement a fully standardized post-interview survey management model to residency didactics. West J
Emerg Med. 2019;21(1):65-70. doi:10.5811/westjem.
for all candidates. Future research should compare
2019.11.44510
website traffic statistics with other measures of
10. Weiss PG, Li STT. Leading change to address the needs
candidate engagement and Match success, including
and well-being of trainees during the COVID-19
satisfaction, wellness, and job performance of resi-
pandemic. Acad Pediatr. 2020;20(6):735-741. doi:10.
dents matched after a virtual recruitment process.
1016/j.acap.2020.06.001
11. National Resident Matching Program. NRMP Program
Conclusions
Results 2018-2022 Specialties Matching Service.
A change management approach was successfully Accessed December 6, 2022. https://www.nrmp.org/
applied to a recruitment redesign project impacting match-data-analytics/fellowship-data-reports/
multiple GME training programs. Using Kotter’s 8 12. Coalition for Physician Accountability. Final Report
steps, we implemented major changes during 2020 and Recommendations for Medical Education
and found increased website engagement with main- Institutions of LCME-Accredited, U.S. Osteopathic,
tained program fill rates over 2 successive recruitment and Non-U.S. Medical School Applicants. Association
cycles. of American Medical Colleges. Accessed November 11,
2022. https://www.aamc.org/media/44731/download
13. Massanelli J, Sexton KW, Lesher CT, et al. Integration
References of web analytics into graduate medical education:
1. Haas MRC, He S, Sternberg K, et al. Reimagining usability study. JMIR Form Res. 2021;5(12):e29748.
residency selection: part 1—a practical guide to doi:10.2196/29748
recruitment in the post-COVID-19 era. J Grad Med
Educ. 2020;12(5):539-544. doi:10.4300/JGME-D-20-
00907.1
All authors are with the Wake Forest University School of
2. Sternberg K, Jordan J, Haas MRC, et al. Reimagining Medicine and Atrium Health Wake Forest Baptist Medical Center.
residency selection: part 2—a practical guide to Matthew C. Miles, MD, MEd, is Associate Professor and Program
Director, Pulmonary and Critical Care Medicine Fellowship; Karl
interviewing in the post-COVID-19 era. J Grad Med M. Richardson, MD, is Assistant Professor, Associate Program
Educ. 2020;12(5):545-549. doi:10.4300/JGME-D-20- Director, Internal Medicine Residency Program; Rachel Wolfe,
00911.1 MD, is Assistant Professor and Program Director, Rheumatology
Fellowship; Kristen Hairston, MD, MPH, is Associate Professor
3. Bernstein SA, Gu A, Chretien KC, Gold JA. Graduate and Program Director, Endocrinology Fellowship; Maryjo
medical education virtual interviews and recruitment Cleveland, MD, is Associate Professor and Program Director,
in the era of COVID-19. J Grad Med Educ. Geriatric Medicine Fellowship; Christopher Kelly, MD, is
Assistant Professor; Jacqueline Lippert, MD, is Assistant
2020;12(5):557-560. doi:10.4300/JGME-D-20- Professor; Nina Mastandrea, BS, is Web and Social Media
00541.1 Manager; and Zachary Pruitt, MD, is Internal Medicine Resident.

Journal of Graduate Medical Education, February 2023 103

EDUCATIONAL INNOVATION

Funding: The authors report no external funding source for this Cagney Gentry; members of the website and digital engagement
study. team Laura Laxton and Allen Gottfried; project manager Ashley
Conflict of interest: The authors declare they have no competing Warlick; Weston & Associates for assisting with creation and
interests. assembly of gift boxes; and Dr. Hal Atkinson, vice-chair for
education and Dr. Gary Rosenthal, chair of medicine, for
The authors would like to acknowledge the valuable providing the initial charge to our group, encouraging us along
contributions of many individuals to this project, including the way, and directing resources to assist our efforts.
program coordinators Shannon Shields, Tammy Davis, Liz
Leonard, Lynne Goulding, Renea Donley, Jana Trevaskis, Amanda Corresponding author: Matthew C. Miles, MD, MEd, Wake Forest
Salmons, Kim Hairston, Alycia Smith, and Heather Crocker-Yates; Baptist Medical Center, [email protected],
program directors Dr. Jason Caldwell, Dr. Prashant Bhave, Dr. Twitter @DrMCMiles
Sujethra Vasu, Dr. Drew MacGregor, Dr. Patrick Green, Dr. Susan
Melin, Dr. Jennifer Davis, Dr. Vera Luther, Dr. Matthew Belford, Dr. Received February 25, 2022; revisions received July 13, 2022, and
Alison Fletcher, and Dr. James Pirkle; videographer Professor November 3, 2022; accepted November 9, 2022.

104 Journal of Graduate Medical Education, February 2023