Faculty Bibliography

BACKGROUND: At Bellevue Hospital Center (BHC), we have a robust Unannounced Standardized Patient (USP) program, where trained actors portraying real patients in the clinical setting, incognito, assess the residents' skills following their visit. We sought to determine the relationship between USP ratings of residents' skills and clinical outcomes among the residents' continuity patient panels to define educationally sensitive patient outcomes. METHODS: We assembled a retrospective cohort of PGY 2 internal medicine residents with at least 2 USP visits between 7/1/14-6/30/15 and ambulatory care patient panels at BHC. The two outcome variables were the percentage of hypertensive patients in the residents' panel with blood pressure (BP) <140/90, and the average of the most recent glycosylated hemoglobin (HbA1C) result among the residents' patients with diabetes. The predictor variables included mean USP ratings of residents' clinical skills and mean faculty rating of the residents' clinic notes (scored for quality on a 0 to 3 scale). USPs used a behaviorally anchored checklist (not done, partly done, well done) for the following domains: communication, case specific assessment, patient education, physical examination, professionalism, management plan, patient satisfaction, and patient activation measure. We tested the correlations between USP scores with BP and HbA1C control, and then developed multivariate, linear regression models of USP scores on BP and HbA1C scores, respectively, each controlling for Avg. Chronic health score (ACHS, derived by scoring different clinical conditions by acuity and used to determine if the panel is getting sicker over time) and total number of patients in the panel (TNPP) because these variables were correlated with both the outcome and predictor variables. RESULTS: 29 PGY 2 residents had a mean of 2.5 (SD 1.0) USP visits during the study period. Residents' patient panels size varied (median 124, range 62- 171) and mean patient age was 48 years (SD 1.4). Patient Activation scores were correlated with Average Chronic Health Score (r = .482, p = .008) and Panel Average last A1c (r = -.311, p = .10). Patient activation scores explained 16% variance in the mean panel last HgA1c, (adjusted R2 .137, p = .08). Case specific Assessment & Patient Education skills across USP cases explained 21.5% of the variance and the Average Chart Note Score explained 14.4% of the variance in % of Hypertension controlled (adjusted R2.378, p < .009). CONCLUSIONS: This exploratory study suggests that learnable resident clinical skills are associated with quality of care indicators for HTN and DM control. In particular, being able to activate patients, assess and educate them and write high quality notes are pathways to quality care. Next steps are to confirm these findings in a larger dataset. Doing so will help align medical education with patient safety and care quality and provide guidance for educational and clinical research aimed at improving the health of populations served

BACKGROUND: It is critically important for medical schools to understand how well prepared their graduates are for residency and yet we do not have a full understanding of how well competencies, assessed in medical school, transfer to residency. This study explores how communication skills measured in a high-stakes, rigorous, comprehensive OSCE in the 3rd year of medical school are related to performance in a similar OSCE in residency and to Residency Program Directors' ratings of intern competence. METHODS: We analyzed communication skills from three time points in a longitudinal cohort of NYU graduates who entered our Internal Medicine Residency (n = 42). 39 provided consent for their GME-UME data to be compiled into a longitudinal, de-identified educational research database through an IRB-approved Registry. Communication skills were measured using a behaviorally anchored 15-item checklist across the 8-station, pass/fail, MS3 OSCE and then midway through PGY2 of residency in a 6- station OSCE (score =% of items rated well done). SPs also provided an overall rating of communication skills (not recommend, with reservations, recommend, highly recommend). In between, at the end of intern year, residents were also rated by their Program Directors on communication skills (and other competencies) using a 4-pt scale. RESULTS: OSCE communication performance assessed in medical school was modestly associated with performance in residency (r = .26, p = .07) but not with Program Directors' ratings of residents' communication skills as interns (r = .11, p = .28). Number of cases in which medical students were "not recommended" for their communication skills was negatively associated with residency OSCE communication scores (r = -.33, p = .05) and positively associated with number of "not recommends" (r = .46, p = .01) but not with Directors' ratings of interns (r = -.08, p = .49). Number of not recommends independently explained more variance in subsequent residency communication scores than did medical school performance (9% vs. 5%). While average OSCE communication scores improved from medical school to residency (65 to 71%), those with 2 or more "not recommends" improved significantly more than those with 1 or no not recommends. Overall, most learners' (21/39) communication scores improved substantially; less than a quarter (7) decreased; and about a quarter (11) were stable. CONCLUSIONS: While communication scores from medical school are associated with similarly measured scores in residency, SPs' decisions to "not recommend" students appear to serve as an independent indicator of future skill deficits. Patterns of change, however, are not necessarily straightforward: students with the most "not recommends" improved the most. The ability to track competency assessments longitudinally is essential for understanding the transition from medical school to residency and future research will benefit from larger sample sizes and the inclusion of learner characteristics that may explain developmental patterns

Introduction Physicians often struggle with the intricacies of brain death determination and communication about end-of-life care. In an effort to remedy this situation, we introduced an educational initiative at our medical school to improve student comprehension and comfort dealing with brain death. Methods Beginning in July 2017, students at our medical school were required to attend a 90-minute brain death didactic and simulation session during their neurology clerkship. Students completed a test immediately before and after participating in the initiative. Results Of the 145 students who participated in this educational initiative between July 2016 and June 2017, 124 (86%) consented to have their data used for research purposes. Students correctly answered a median of 53% of questions (IQR 47-58%) on the pretest and 86% of questions (IQR 78-89%) on the posttest (p<0.001). Comfort with both performing a brain death evaluation and talking to a family about brain death improved significantly after this initiative (18% of students were comfortable performing a brain death evaluation before the initiative and 86% were comfortable doing so after the initiative, p<0.001; 18% were comfortable talking to a family about brain death before the initiative and 76% were comfortable doing so after the initiative, p<0.001). Conclusions Incorporation of simulation in undergraduate medical education is high-yield. At our medical school, knowledge about brain death and comfort performing a brain death exam or talking to a family about brain death was poor prior to development of this initiative, but awareness and comfort dealing with brain death improved significantly after this initiative. This initiative was clearly a success and can serve as a model for brain death education at other medical schools

OBJECTIVES: Professionalism education is a vital component of surgical training. This research attempts to determine whether an annual, year-long professionalism curriculum in a large surgical residency can effectively change professionalism attitudes. SUMMARY OF BACKGROUND DATA: The ACGME mandated 6 competencies in 2003. The competencies of Professionalism and Interpersonal/Professional Communication Skills had never been formally addressed in surgical resident education in the past. METHODS: A professionalism curriculum was developed focusing on specific resident professionalism challenges: admitting mistakes, effective communication with colleagues at all levels, delivering the news of an unexpected death, interdisciplinary challenges of working as a team, the cultural challenge of obtaining informed consent through an interpreter, and the stress of surgical practice on you and your family. These professionalism skills were then evaluated with a 6-station Objective Structured Clinical Examination (OSCE). Identical OSCE scenarios were administered to 2 cohorts of surgical residents: in 2007 (before instituting the professionalism curriculum in 2008) and again in 2014. Surgical residents were rated by trained Standardized Patients according to a behaviorally anchored professionalism criteria checklist. RESULTS: An analysis of variance was conducted of overall OSCE professionalism scores (% well done) as the dependent variable for the 2 resident cohorts (2007 vs 2014). The 2007 residents received a mean score of 38% of professionalism items "well done" (SD 9%) and the 2014 residents received a mean 59% "well done" (SD 8%). This difference is significant (F = 49.01, P < .001). CONCLUSIONS: Professionalism education has improved surgical resident understanding, awareness, and practice of professionalism in a statistically significant manner from 2007 to 2014. This documented improvement in OSCE performance reflects the value of a professionalism curriculum in the care of the patients we seek to serve.

BACKGROUND: Interprofessional collaboration (IPC) is essential for quality care. Understanding residents' level of competence is a critical first step to designing targeted curricula and workplace learning activities. In this needs assessment, we measured residents' IPC competence using specifically designed Objective Structured Clinical Exam (OSCE) cases and surveyed residents regarding training needs. METHODS: We developed three cases to capture IPC competence in the context of physician-nurse collaboration. A trained actor played the role of the nurse (Standardized Nurse - SN). The Interprofessional Education Collaborative (IPEC) framework was used to create a ten-item behaviorally anchored IPC performance checklist (scored on a three-point scale: done, partially done, well done) measuring four generic domains: values/ethics; roles/responsibilities; interprofessional communication; and teamwork. Specific skills required for each scenario were also assessed, including teamwork communication (SBAR and CUS) and patient-care-focused tasks. In addition to evaluating IPC skills, the SN assessed communication, history-taking and physical exam skills. IPC scores were computed as percent of items rated well done in each domain (Cronbach's alpha > 0.77). Analyses include item frequencies, comparison of mean domain scores, correlation between IPC and other skills, and content analysis of SN comments and resident training needs. RESULTS: One hundred and seventy-eight residents (of 199 total) completed an IPC case and results are reported for the 162 who participated in our medical education research registry. IPC domain scores were: Roles/responsibilities mean = 37 % well done (SD 37 %); Values/ethics mean = 49 % (SD 40 %); Interprofessional communication mean = 27 % (SD 36 %); Teamwork mean = 47 % (SD 29 %). IPC was not significantly correlated with other core clinical skills. SNs' comments focused on respect and IPC as a distinct skill set. Residents described needs for greater clarification of roles and more workplace-based opportunities structured to support interprofessional education/learning. CONCLUSIONS: The IPC cases and competence checklist are a practical method for conducting needs assessments and evaluating IPC training/curriculum that provides rich and actionable data at both the individual and program levels.