Faculty Bibliography

PROBLEM: Efforts to evaluate and optimize the effectiveness of medical education have been limited by the difficulty of designing medical education research. Longitudinal, epidemiological views of educational outcomes can help overcome limitations, but these approaches require "bigger data"-more learners, sources, and time points. The rich data institutions collect on students and residents can be mined, however, ethical and practical barriers to using these data must first be overcome. APPROACH: In 2008, the authors established the Research on Medical Education Outcomes (ROMEO) Registry, an educational data registry modeled after patient registries. New York University School of Medicine students, residents, and fellows provide consent for routinely collected educational, performance, quality improvement, and clinical practice data to be compiled into a deidentified, longitudinal database. As of January 2015, this registry included 1,225 residents and fellows across 12 programs (71% consent rate) and 841 medical students (86% consent rate). Procedures ensuring voluntary informed consent are essential to ethical enrollment and data use. Substantial resources are required to provide access to and manage the data. OUTCOMES: The registry supports educational scholarship. Seventy-two studies using registry data have been presented or published. These focus on evaluating the curriculum, quality of care, and measurement quality and on assessing needs, competencies, skills development, transfer of skills to practice, remediation patterns, and links between education and patient outcomes. NEXT STEPS: The authors are working to integrate assessment of relevant outcomes into the curriculum, maximize both the quantity and quality of the data, and expand the registry across institutions.

BACKGROUND: Interprofessional collaboration (IPC) is essential for quality care. Understanding residents' level of competence is a critical first step to designing targeted curricula and workplace learning activities. In this needs assessment, we measured residents' IPC competence using specifically designed Objective Structured Clinical Exam (OSCE) cases and surveyed residents regarding training needs. METHODS: We developed three cases to capture IPC competence in the context of physician-nurse collaboration. A trained actor played the role of the nurse (Standardized Nurse - SN). The Interprofessional Education Collaborative (IPEC) framework was used to create a ten-item behaviorally anchored IPC performance checklist (scored on a three-point scale: done, partially done, well done) measuring four generic domains: values/ethics; roles/responsibilities; interprofessional communication; and teamwork. Specific skills required for each scenario were also assessed, including teamwork communication (SBAR and CUS) and patient-care-focused tasks. In addition to evaluating IPC skills, the SN assessed communication, history-taking and physical exam skills. IPC scores were computed as percent of items rated well done in each domain (Cronbach's alpha > 0.77). Analyses include item frequencies, comparison of mean domain scores, correlation between IPC and other skills, and content analysis of SN comments and resident training needs. RESULTS: One hundred and seventy-eight residents (of 199 total) completed an IPC case and results are reported for the 162 who participated in our medical education research registry. IPC domain scores were: Roles/responsibilities mean = 37 % well done (SD 37 %); Values/ethics mean = 49 % (SD 40 %); Interprofessional communication mean = 27 % (SD 36 %); Teamwork mean = 47 % (SD 29 %). IPC was not significantly correlated with other core clinical skills. SNs' comments focused on respect and IPC as a distinct skill set. Residents described needs for greater clarification of roles and more workplace-based opportunities structured to support interprofessional education/learning. CONCLUSIONS: The IPC cases and competence checklist are a practical method for conducting needs assessments and evaluating IPC training/curriculum that provides rich and actionable data at both the individual and program levels.

PURPOSE/OBJECTIVE:For accelerated 3-year MD (3YMD) pathways to be fully adopted in medical education, a comprehensive analysis of outcome data is needed. This study includes 7 accelerated 3YMD graduating classes at NYU Grossman School of Medicine (NYUGSOM) and reports on outcomes from both medical school and internship compared to their 4-year MD (4YMD) counterparts. METHOD/METHODS:Outcomes across the undergraduate-graduate medical education continuum for the first 7 classes of NYUGSOM graduates (matriculated from 2013-2019) from the accelerated 3YMD (n = 136) and 4YMD pathways (n = 681) were compared. For the internship outcomes, 3YMD interns were compared with 4YMD interns who graduated from NYUGSOM and all 4YMD interns (4YMD graduates from NYUGSOM and any other medical school) at NYUGSOM residencies. RESULTS:Accelerated 3YMD students were approximately 5 months older at admission and had higher multiple mini-interview scores than 4YMD students. Overall, accelerated 3YMD students performed similarly to 4YMD students during medical school and internship. Significant differences included higher performance by 3YMD students on preclerkship exams and lower performance on Steps 1 and 2 (average: 5.6 and 8.3 fewer points, respectively) and the physical examination portion of the NYUGSOM Comprehensive Clinical Skills Exam. Internship data indicated comparable team assessments across all residencies, statistically significant higher performance on Step 3 when compared to all 4YMD interns, and, in internal medicine, comparable clinical reasoning between 3YMD and all 4YMD interns. When comparing 3YMD interns to all 4YMD interns in the internal medicine residency program, 3YMD interns had a statistically significantly higher performance on milestones. CONCLUSIONS:The outcomes from 7 years of graduating accelerated 3YMD students at NYUGSOM show similar performance in medical school and early residency to 4YMD graduates. Long-term study of accelerated 3YMD students from NYUGSOM and other medical schools is needed to further validate the success of this innovative medical education pathway.

BACKGROUND:Suboptimal support for colleagues experiencing discrimination can adversely impact clinician well-being and patient care. AIM/OBJECTIVE:To describe resident performance and experience during an Objective Structured Clinical Examination (OSCE) case centered on supporting a trainee facing discrimination to inform enhanced, supportive learning environments. SETTING/METHODS:Formative, internal medicine OSCE at a simulation center. PARTICIPANTS/METHODS:148 second-year residents across 2018, 2019, 2021, 2022. PROGRAM DESCRIPTION/METHODS:Residents had 10 min to support a Muslim standardized intern (SI) experiencing discrimination from a patient. The SI rated resident performance across Supervision, Relationship Development, and Support domains and provided written feedback. Post-OSCE evaluations elicited resident reflections on case challenges. PROGRAM EVALUATION/RESULTS:Proficient residents (≥ 80% average score across domains, n = 85) performed better in all items, except in not acting defensive and collaborating with SI to develop follow-up plan, compared to non-proficient residents (n = 65). The SI described effective approaches to feeling supported, including using empathetic statements, stating personal stance on discrimination, exhibiting supportive body language, and verbalizing support. Stating knowledge of situation upfront was an area of improvement. Residents found engaging the distressed SI difficult. DISCUSSION/CONCLUSIONS:Use of an explicit discrimination OSCE case can help identify effective approaches to supporting targets of discriminatory patients to inform future training.

BACKGROUND:Coaching has been proposed to support the transition to residency. Clarifying its impact will help define its value and best use. OBJECTIVE:To explore the experiences of residents working with coaches through the residency transition. DESIGN/METHODS:A cohort comparison survey compared experiences of a coached resident cohort with coaches to the prior, uncoached cohort. PARTICIPANTS/METHODS:Post-graduate year (PGY)-2 residents in internal medicine, obstetrics and gynecology, emergency medicine, and pathology at a single academic center. INTERVENTIONS/METHODS:Faculty trained as coaches had semi-structured meetings with graduating medical students and residents throughout the PGY-1 year. MAIN MEASURES/METHODS:An online anonymous survey assessed effects of coaching on measures of self-directed learning, professional development, program support and impact of coaching using existing scales (2-item Maslach Burnout Inventory, Brief Resilient Coping Scale, 2-item Connor-Davidson Resilience Scale, Stanford Professional Fulfillment Inventory), and novel measures adapted for this survey. Bivariate analyses (t-tests and chi-square tests) compared cohort responses. MANOVA assessed the effects of coaching, burnout and their interactions on the survey domains. KEY RESULTS/RESULTS:Of 156 PGY2 residents, 86 (55%) completed the survey. More residents in the "un-coached" cohort reported burnout (69%) than the "coached" cohort (51%). Burnout was significantly and negatively associated (F = 3.97 (df 7, 75); p < .001) with the learning and professional development outcomes, while being coached was significantly and positively associated with those outcomes (F = 5.54 (df 9, 75); p < .001). Significant interaction effects were found for goal-setting attitudes, professional fulfillment, and perceived program career support such that the positive differences in these outcomes between coached and un-coached residents were greater among burned out residents. Coached residents reported a positive impact of coaching across many domains. CONCLUSIONS:Residents experiencing coaching reported better professional fulfillment and development outcomes, with more pronounced differences in trainees experiencing burnout. Coaching is a promising tool to support a fraught professional transition.

BACKGROUND:Unannounced standardized patients (USPs) have long been used to measure clinical performance in situ. These incognito actors capture data on clinician skills during an encounter, as well as patient experience more broadly. A robust USP program requires extensive preparation and standardization efforts. Given the widespread expansion of USPs for education, research, and improvement efforts, we conducted a systematic review with the goal of capturing the breadth of uses of USPs across settings, along with the standardization measures employed across studies. METHODS:In collaboration with a medical librarian, we conducted systematic searches across six databases. Two independent researchers screened each report for inclusion. Three coders extracted and reviewed study characteristics and data from the studies deemed eligible for inclusion. We extracted data on: target population, setting, and assessed skills. We also captured the reliability and fidelity measures described in each study, including USP detection, USP training methods, and assessment measures. RESULTS:128 articles were included. Individual clinicians were the most frequently targeted population (n = 114, 89 %). Common clinician roles included physicians (n = 92, 72 %) and pharmacists (n = 12, 9 %). The collective care team was the target in two studies (2 %), and systems and larger healthcare facilities were targeted in only 1 (1 %) and 13 (10 %) studies, respectively. Studies were primarily conducted in ambulatory settings (n = 118, 92 %). History gathering (n = 76, 59 %), communication (n = 55, 43 %), counseling (n = 51, 40 %), and patient education (n = 49, 38 %) were commonly assessed, as were correct diagnosis (n = 34, 27 %), appropriate ordering of labs/tests (n = 30, 23 %), referrals (n = 35, 27 %), and prescriptions (n = 36, 28 %). USP detection reporting was variable across studies; however, no detection information was provided for 48 studies. 62 % of articles reported incorporating a measure of reliability or fidelity into their study, while the remainder either failed to provide adequate information on use of these measures. CONCLUSIONS:We explored USP use across settings and describe the scope and limitations of the literature. USPs capture a range of data domains but a lack uniform report of reliability measures can potentially undermine findings. Future studies should incorporate and uniformly report out on detection, training, and assessment.

PURPOSE/OBJECTIVE:The aim of this study was to identify successful recruitment strategies and obstacles reported by principal investigators (PIs) of the Zoster Eye Disease Study (ZEDS). METHODS:A web-based survey was created by a subset of ZEDS PIs and distributed to ZEDS PIs after study enrollment was closed. The survey queried investigators about recruitment strategies and obstacles, use of prophylactic oral antiviral medication, electronic medical records, telemedicine, COVID-19 effect, turnover of research staff, and recruitment outreach to minority and underserved populations. The survey allowed objective measures and free-text options. Analysis from centers with higher enrollment was compared with centers with lower enrollment to identify successful strategies and common obstacles. RESULTS:The most frequently cited helpful strategies were participation from ophthalmologists within the PI's institution (33/63, 52%), ophthalmology resident referrals (23/63, 37%), and chart review (23/63, 37%). Travel to participating clinical center (42/63, 67%) and ongoing prophylactic use of oral systemic antiviral medication (39/63, 62%) were common obstacles. Research coordinator turnover was identified as a contributor to reduced recruitment success by 49% (31/63) of PIs and made recruitment much harder for 22% (14/63). A small number of investigators used telemedicine (18/63, 29%) and few made efforts to recruit from minority and underserved populations (10/63, 16%). CONCLUSIONS:Our survey highlights the importance of internal ophthalmologist referral, chart review, and research coordinator commitment for successful clinical trial recruitment. We discuss the impact of prophylactic use of oral antiviral medication on recruitment. Future randomized clinical trials should mobilize the helpful recruitment strategies and improve outreach to underserved populations.

Over the last decade there has been tremendous growth in the development of accelerated MD pathways that allow medical students to graduate in three years. Developing an accelerated pathway program requires commitment from students and faculty with intensive re-thinking and altering of the curriculum to ensure adequate content to achieve competency in an accelerated timeline. A re-visioning of assessment and advising must follow and the application of AI and new technologies can be added to support teaching and learning. We describe the curricular revision to an accelerated pathway at NYU Grossman School of Medicine highlighting our thought process, conceptual framework, assessment methods and outcomes over the last ten years.

OBJECTIVES/OBJECTIVE:Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS:) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS:Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS/CONCLUSIONS:This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.