Faculty Bibliography

IMPORTANCE/UNASSIGNED:Antibiotic prescribing varies across primary care physicians (PCPs) and disproportionately harms older adults, yet whether residency training is a factor in this variation is unknown. OBJECTIVE/UNASSIGNED:To evaluate whether PCPs' Accreditation Council for Graduate Medical Education (ACGME) competency ratings toward the end of residency are associated with their early-career pattern of antibiotic prescribing to older adults. DESIGN, SETTING, AND PARTICIPANTS/UNASSIGNED:This cohort study involved graduates of US ACGME-accredited internal medicine and family medicine residency programs between January 1, 2015, and December 31, 2020, who were in their first 5 years of primary care practice. Their ACGME Milestone ratings (hereafter competency ratings) were linked to Medicare Parts B and D public claims from January 1, 2015, to December 31, 2022. Data analysis was conducted between March 2024 and May 2026. EXPOSURE/UNASSIGNED:Overall competency rating in the middle of the final year of residency. Ratings for 22 subcompetencies (5-point developmental scale) were averaged within 6 core competency domains (patient care, medical knowledge, professionalism, interpersonal and communication skills, systems-based practice, and practice-based learning and improvement), averaged across domains, and then standardized at the residency program level. MAIN OUTCOME AND MEASURE/UNASSIGNED:Annual antibiotic prescription rate. This primary outcome was expressed as an incidence rate ratio (IRR) adjusted for physician, residency program, practice location, and patient-panel factors. Residency training program was examined as a secondary factor. RESULTS/UNASSIGNED:The analysis included 31 800 PCPs (mean [SD] age on practice entry, 32.8 [4.2] years; 15 990 females [50%]) who graduated from 1090 residency programs, representing 129 225 physician-years of prescribing data. Among these physicians, 58% graduated from a US medical school, 90% earned an MD degree, and 51% specialized in internal medicine and 49% specialized in family medicine. Each physician panel had a mean (SD) of 169 (144) Medicare beneficiaries per year (mean [SD] age, 70.3 [3.5] years; 59% female patients). Physicians prescribed a mean (SD) of 54.1 (52.2) antibiotics per year, and the full cohort wrote 7 664 372 antibiotic prescriptions across the study period. In unadjusted analyses, graduates from internal medicine and family medicine residency programs with highest vs lowest quintile prescribing rates prescribed a difference of 14.9 (95% CI, 13.8-15.9; P < .001) and 18.8 (95% CI, 17.9-19.7; P < .001) additional antibiotics per 100 beneficiaries per year, respectively. In adjusted analysis, each 1-SD increase in overall competency rating was associated with 0.7 (95% CI, 0.5-0.9) fewer antibiotic prescriptions per 100 beneficiaries per year (adjusted IRR, 0.98; 95% CI, 0.97-0.99; P < .001). Each year of experience was associated with higher antibiotic prescribing (IRR, 1.01; 95% CI, 1.01-1.02; P < .001), even accounting for the reduction in prescribing that occurred across study years. CONCLUSIONS AND RELEVANCE/UNASSIGNED:This study found that higher assessed competence near the end of residency was associated with lower antibiotic prescribing in early primary care practice. Competency assessments may inform prescribing variation-focused efforts during training and early practice.

PURPOSE/OBJECTIVE:To develop Trainee Attributable & Automatable Care Evaluations in Real‑Time (TRACERs) for inpatient diabetes management, collect validity evidence for their use in formative assessment, and explore performance variation across residents and institutions. METHOD/METHODS:In 2023, a multi‑institutional team created two TRACERs based on type 2 diabetes guidelines-discourage bolus‑only insulin (TRACER #1) and encourage basal (± bolus) insulin (TRACER #2)-for internal medicine residents at three large residency programs. Residents were attributed to inpatient admissions based on placing the most medication orders in the first 12 hours. Structured queries extracted 35 discrete variables from the electronic health record (EHR). Two experts per institution reviewed random admissions (July-August 2022) to establish criterion validity. A retrospective cohort (July 2020-June 2023) added validity evidence. RESULTS:Automated extraction achieved ≥96% sensitivity and ≥95% specificity when compared to manual review. Among 615 residents attributed to 6,192 admissions of patients with type 2 diabetes at high risk for hyperglycemia, TRACER #1 occurred in 42.6% (1,689/3,965) of admissions at Program A, 28.9% (408/1,410) at Program B, and 26.7% (218/817) at Program C. TRACER #2 occurred in 44.9% (367/817) of Program C admissions versus 24.2% (959/3,965) at Program A and 28.2% (397/1,410) at Program B (all P < .001). Four resident-level insulin‑ordering profiles were identified-consistent basal-bolus insulin use (most guideline-concordant), basal-predominant, bolus-predominant, and bolus-only (most guideline-discordant)-with between‑resident variation exceeding between‑program differences. Longitudinally, cohort-level trends masked opposing individual trajectories-some residents improved with exposure while others worsened-and performance tertiles were distinguishable early in training. CONCLUSIONS:TRACERs revealed substantial institution‑ and resident‑level variation in insulin‑ordering practices, including guideline deviations, demonstrating potential for real‑time formative feedback. Multi‑institutional implementation highlighted scalability barriers, including EHR heterogeneity and workflow‑dependent attribution, underscoring the need for continued refinement and broader validation.

Acute coronary syndrome (ACS) requires prompt treatment, yet management delays are difficult to identify. In this study, we developed a large language model (LLM) system to identify ACS management delays and characterized delay cases. Admissions to internal medicine residents at NYU July 2022-June 2025 (n=4,642) were included. Prompts were validated to determine if the resident admission note documented initiation of ACS management and if the initial cardiology consult note documented initiation of ACS management (ground truth) (n=161 for each). Discordant cases were reviewed by three physicians using a validated tool to confirm management delays. Demographics and key clinical findings of patients with and without delays were compared. The LLM identified management delays with a 52% positive predictive value (n=35/67). Patients who were older, females, and with preferred language other than English or Spanish were more likely to have a management delay (73.4 ± 15.3 vs 68.5 ± 12.6 years-old, p=0.036, 56.8% vs 34% females, p=0.014, and 27.0% vs 15.5% other preferred language, p=0.046, in management delay vs non-management delay cases). The management delay group had longer average time in hours to receiving heparin, aspirin, and cardiac catheterization (56.91 ± 56.78 vs 18.97 ± 13.76, p<0.001, 13.94 ± 16.64 vs 8.23 ± 9.82, p=0.005, and 65.12 ± 51.65 vs 39.51 ± 44.19, p=0.006, respectively in management delay vs non-management delay cases). In conclusion, the LLM-based system we developed to identify ACS management delays can detect cases at scale to inform individual and systems-level interventions to improve quality of ACS care.

BACKGROUND:Poor documentation quality can significantly affect healthcare operations, but the feedback process for clinicians to improve clinical notes is time-consuming and often insufficient. Large language models (LLMs) such as Generative Pre-trained Transformer 4 (GPT-4) have the potential to streamline this process. OBJECTIVES/OBJECTIVE:To determine whether an LLM can generate feedback to improve the medical contingency and discharge planning (MCDP) component of clinical documentation that is non-inferior to feedback by physicians. METHODS:A cross-sectional study of GPT-4 feedback and physician feedback on inpatient progress notes was conducted. A random sample of 64 inpatient progress notes identified by the validated AI Audit Tool as having a low likelihood of containing MCDP was included from adult general medicine patients hospitalized at New York University Langone Health (NYULH) in December 2023. Both GPT-4 model and attending physicians generated feedback on these inpatient progress notes. A/B testing was then conducted on the measures of understandability, usefulness, acceptability, and impartiality. Evaluations employed 5-point Likert scales that were converted to 10-point bidirectional interval scales for interpretability, ranging from -10 (human suggestions significantly better) to +10 (GPT-4 suggestions significantly better), with a non-inferiority threshold set to -1 for the primary endpoint. RESULTS:64 inpatient progress notes were included, representing 55% female patients with a median age of 73. GPT-4 feedback was non-inferior to physician feedback in all measures: understandability (mean 1.27, 95% CI 0.73 to 1.8, P < 0.001), usefulness (mean 2.09, 95% CI 1.27 to 2.91, P < 0.001), acceptability (mean 2.07, 95% CI 1.33 to 2.81, P < 0.001), and impartiality (mean -0.20, 95% CI -0.52 to 0.12, P < 0.001). CONCLUSIONS:This study shows that an LLM can be leveraged to generate note quality feedback that is non-inferior to expert clinician feedback.

IMPORTANCE/UNASSIGNED:Limited research exists on sex, racial, and ethnic disparities in required graduate medical education (GME) resident competency ratings across specialties during sensitive periods when career decision-making occurs. Rating disparities using an antideficit-based approach measured by exceptionally high ratings are underexplored in GME. OBJECTIVE/UNASSIGNED:To assess the association of exceptionally high ratings in the Accreditation Council for Graduate Medical Education (ACGME) Milestones during time-sensitive training periods across specialties with differences among residents' characteristics, including sex, race, and ethnicity. DESIGN, SETTING, AND PARTICIPANTS/UNASSIGNED:This cross-sectional analysis was conducted between March 15 and December 31, 2025, using 2018 to 2021 Association of American Medical Colleges and ACGME data. Postgraduate year (PGY) 2 residents training at US ACGME-accredited emergency medicine, family medicine, internal medicine, obstetrics and gynecology, pediatrics, and surgery residency programs between 2018 and 2021 who self-reported sex, race, or ethnicity were studied. EXPOSURE/UNASSIGNED:Required Milestones ratings at the end of PGY-2 training associated with resident sex and race or ethnicity (underrepresented in medicine [URiM] and Asian), while controlling for preresidency Step 2 Clinical Knowledge examination scores. MAIN OUTCOMES AND MEASURES/UNASSIGNED:Proportion and adjusted odds ratios (AORs) for exceptionally high resident-level ratings (80th percentile level) across competencies in interpersonal and communication skills, medical knowledge, patient care, practice-based learning and improvement, professionalism, and systems-based practice. RESULTS/UNASSIGNED:Among 19 492 PGY-2 residents across 1754 programs, 10 384 (53.3%) were female, 28 (0.14%) American Indian or Alaskan Native, 4327 (22.2%) Asian, 1106 (5.7%) Black, 1008 (5.2%) Hispanic or Latinx, 3 (0.02%) Native Hawaiian or Pacific Islander, 12 269 (62.9%) White, 751 (3.9%) reporting 2 or more races, and 3423 (17.6%) classified as URiM. Exceptional rating differences were identified by sex, race, and ethnicity. Across all specialties, female residents had greater odds for 80th percentile ratings (AOR, 1.12; 95% CI, 1.05-1.21; P < .001); whereas when compared with White residents, URiM residents (AOR, 0.68; 95% CI, 0.62-0.76; P < .001) and Asian residents (AOR, 0.67; 95% CI, 0.60-0.74; P < .001) were less likely to have 80th percentile ratings than White residents. Within specialties, URiM residents in emergency medicine, family medicine, internal medicine, obstetrics and gynecology, and surgery were less likely to have 80th percentile ratings, whereas Asian residents in family medicine, internal medicine, pediatrics, and surgery were also less likely than White residents. CONCLUSION AND RELEVANCE/UNASSIGNED:In this cross-sectional national study of residents, exceptionally higher ratings were associated with differing resident characteristics during crucial career planning phases. These results suggest the need for more studies to explore factors of resident success during GME training.

There is limited data on which hospitalist switch day is optimal for hospital operations and throughput. A quality improvement intervention was implemented, changing the hospitalist switch day from Monday to Tuesday. Retrospective observational analysis revealed an increase in Monday discharges (1.3%, p = .01), a decrease in Tuesday discharges (-1.6%, p < .005), and a significant reduction in 30-day unplanned readmission rates (-1.5%, p = .003), with no significant changes in the average length of stay. Additional studies are needed to further verify these findings in different hospital settings and to consider other switch day patterns.

BACKGROUND:Delayed diagnosis of venous thromboembolism (VTE) is prevalent among hospitalized patients, yet case identification is challenging and feedback limited. OBJECTIVE:To develop a large language model (LLM)-based electronic-trigger to identify VTE diagnostic delays. METHODS:All admissions to internal medicine (IM) residents at NYU Langone Health between January 2022 and December 2023 (n = 20,843) were included. Using an open-source LLM, prompts were validated to detect (1) residents considering VTE in admission notes and (2) VTE confirmation in five types of imaging reports (n = 100 for each prompt validation set). The validated prompts were applied to determine discordance between admission note differential omitting VTE and imaging report confirming VTE. Two hospitalists reviewed discordant cases using a validated tool to identify diagnostic delays. Hospitalizations were labeled as diagnostic delays, in-hospital complication, or false-positive. Based on in-hospital complication and false-positive patterns, exclusion criteria were implemented. Positive predictive value (PPV) and negative predictive value (NPV) were calculated. RESULTS:The LLM prompts correctly classified admission notes and VTE imaging studies with high accuracy (range 98%-100%, n = 699 VTE cases identified). Of the 137 diagnostic delays the LLM-based electronic-trigger identified, 31 were true-positives, 60 in-hospital complications, and 46 false-positives. 4.4% of all VTE hospitalizations had a diagnostic delay. With the exclusion criteria, the PPV was 48% (95% confidence interval [CI], 35%-62%) and NPV was 95% (95% CI, 87%-98%). CONCLUSIONS:We developed the first LLM-based electronic-trigger to identify VTE diagnostic delays, with higher performance than existing non-LLM electronic-triggers. LLM-based approaches can facilitate diagnostic performance feedback and are scalable to other conditions and institutions.