Faculty Bibliography

PURPOSE:Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review. METHOD:Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations. RESULTS:The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review. CONCLUSIONS:The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods.

Background/UNASSIGNED:Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged. Objective/UNASSIGNED:This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process. Methods/UNASSIGNED:Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories. Results/UNASSIGNED:Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13). Conclusions/UNASSIGNED:This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change.

The 2019 novel coronavirus (COVID-19) pandemic has led to dramatic changes in the 2020 residency application cycle, including halting away rotations and delaying the application timeline. These stressors are laid on top of a resident selection process already under duress with exploding application and interview numbers-the latter likely to be exacerbated with the widespread shift to virtual interviewing. Leveraging their trainee perspective, the authors propose enforcing a cap on the number of interviews that applicants may attend through a novel interview ticket system (ITS). Specialties electing to participate in the ITS would select an evidence-based, specialty-specific interview cap. Applicants would then receive unique electronic tickets-equal in number to the cap-that would be given to participating programs at the time of an interview, when the tickets would be marked as used. The system would be self-enforcing and would ensure each interview represents genuine interest between applicant and program, while potentially increasing the number of interviews-and thus match rate-for less competitive applicants. Limitations of the ITS and alternative approaches for interview capping, including an honor code system, are also discussed. Finally, in the context of capped interview numbers, the authors emphasize the need for transparent preinterview data from programs to inform applicants and their advisors on which interviews to attend, learning from prior experiences and studies on virtual interviewing, adherence to best practices for interviewing, and careful consideration of how virtual interviews may shift inequities in the resident selection process.

Holistic review of residency applications is touted as the gold standard for selection, yet vast application numbers leave programs reliant on screening using filters such as United States Medical Licensing Examination scores that do not reliably predict resident performance and may threaten diversity. Applicants struggle to identify which programs to apply to, and devote attention to these processes throughout most of the fourth year, distracting from their clinical education. In this perspective, educators across the undergraduate and graduate medical education continuum propose new models for student-program compatibility based on design thinking sessions with stakeholders in obstetrics and gynecology education from a broad range of training environments. First, we describe a framework for applicant-program compatibility based on applicant priorities and program offerings, including clinical training, academic training, practice setting, residency culture, personal life, and professional goals. Second, a conceptual model for applicant screening based on metrics, experiences, attributes, and alignment with program priorities is presented that might facilitate holistic review. We call for design and validation of novel metrics, such as situational judgment tests for professionalism. Together, these steps could improve the transparency, efficiency and fidelity of the residency application process. The models presented can be adapted to the priorities and values of other specialties.

Introduction: The role of medical students in catalyzing and leading curricular change in US medical schools is not well described. Here, American Medical Association student and physician leaders in the Accelerating Change in Medical Education initiative use qualitative methods to better define student leadership in curricular change.Methods: The authors developed case studies describing student leadership in curricular change efforts. Case studies were presented at a national medical education workshop; participants provided worksheet reflections and were surveyed, and responses were transcribed. Kotter's change management framework was used to categorize reported student roles in curricular change. Thematic analysis was used to identify barriers to student engagement and activators to overcome these barriers.Results: Student roles spanned all eight steps of Kotter's change management framework. Barriers to student engagement were related to faculty (e.g. view student roles narrowly), students (e.g. fear change or expect faculty-led curricula), or both (e.g. lack leadership training). Activators were: (1) recruiting collaborative faculty, staff, and students; (2) broadening student leadership roles; (3) empowering student leaders; and (4) recognizing student successes.Conclusions: By applying these activators, medical schools can build robust student-faculty partnerships that maximize collaboration, moving students beyond passive educational consumption to change agency and curricular co-creation.

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

PURPOSE:The extent of medical trainees' engagement in scholarly medical education publication is not well described. This study sought to quantify the prevalence of medical student- and resident-authored medical education publications over 15 years, a benchmark essential for understanding current and future trends in trainee scholarship. METHOD:Of 91 identified journals, 16 met inclusion criteria as indexed general medical education journals. Only Academic Medicine provided complete author role information, allowing identification of medical student and resident authors. The authors retrospectively compiled and analyzed citation records from Academic Medicine from 2002 to 2016, tracking trainee authorship, author position, and publication type. RESULTS:A total of 6,280 publications were identified, of which 4,635 publications, by 16,068 authors, met inclusion criteria. Trainees were 6.0% (966/16,068) of all authors and authored 14.5% (673/4,635) of all publications. Trainee authorship rates varied by publication type: Trainees authored 33.3% (160/480) of medical humanities publications versus 6.9% (27/392) of commentaries. From 2002-2004 to 2014-2016, the proportion of authors who were trainees increased from 3.9% (73/1,853) to 7.1% (330/4,632) (P < .001 for trend). Over the same period, the percentage of trainee-authored publications increased: 9.4% (58/620) to 18.8% (225/1,199) (P < .001 for trend), driven primarily by increased trainee first authorship. CONCLUSIONS:Trainees constitute a small but growing proportion of authors and authored publications in Academic Medicine. Further work is needed to understand what trainee-, institutional-, and journal-level factors contribute to this trend, and whether similar increases in trainee authorship are occurring in other journals and fields.

INTRODUCTION:The United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) are important for trainee medical knowledge assessment and licensure, medical school program assessment, and residency program applicant screening. Little is known about how USMLE performance varies between institutions. This observational study attempts to identify institutions with above-predicted USMLE performance, which may indicate educational programs successful at promoting students' medical knowledge. METHODS:Self-reported institution-level data was tabulated from publicly available US News and World Report and Association of American Medical Colleges publications for 131 US allopathic medical schools from 2012-2014. Bivariate and multiple linear regression were performed. The primary outcome was institutional mean USMLE Step 1 and Step 2 CK scores outside a 95% prediction interval (≥2 standard deviations above or below predicted) based on multiple regression accounting for students' prior academic performance. RESULTS:Eighty-nine US medical schools (54 public, 35 private) reported complete USMLE scores over the three-year study period, representing over 39,000 examinees. Institutional mean grade point average (GPA) and Medical College Admission Test score (MCAT) achieved an adjusted R2 of 72% for Step 1 (standardized βMCAT 0.7, βGPA 0.2) and 41% for Step 2 CK (standardized βMCAT 0.5, βGPA 0.3) in multiple regression. Using this regression model, 5 institutions were identified with above-predicted institutional USMLE performance, while 3 institutions had below-predicted performance. CONCLUSIONS:This exploratory study identified several US allopathic medical schools with significant above- or below-predicted USMLE performance. Although limited by self-reported data, the findings raise questions about inter-institutional USMLE performance parity, and thus, educational parity. Additional work is needed to determine the etiology and robustness of the observed performance differences.

New instructional technologies have been increasingly incorporated into the medical school learning environment, including lecture video recordings as a substitute for live lecture attendance. The literature presents varying conclusions regarding how this alternative experience impacts students' academic success. Previously, a multi-year study of the first-year medical histology component at the University of Michigan found that live lecture attendance was positively correlated with learning success, while lecture video use was negatively correlated. Here, three cohorts of first-year medical students (N = 439 respondents, 86.6% response rate) were surveyed in greater detail regarding lecture attendance and video usage, focusing on study behaviors that may influence histology learning outcomes. Students who reported always attending lectures or viewing lecture videos had higher average histology scores than students who employed an inconsistent strategy (i.e., mixing live attendance and video lectures). Several behaviors were negatively associated with histology performance. Students who engaged in "non-lecture activities" (e.g., social media use), students who reported being interrupted while watching the lecture video, or feeling sleepy/losing focus had lower scores than their counterparts not engaging in these behaviors. This study suggests that interruptions and distractions during medical learning activities-whether live or recorded-can have an important impact on learning outcomes. Anat Sci Educ 11: 366-376. © 2017 American Association of Anatomists.