Faculty Bibliography

BACKGROUND: Entrustable Professional Activities (EPAs) and milestones are expert judgments made based on many formative assessments. Their validity is dependent on the number of assessments but attention is increasingly being paid to having a "fair" sample of observations equally distributed across residents and contexts. OSCEs provide such a consistent, fair sample of behavior assessed under controlled conditions but have mostly used been used to provide granular skills feedback. We explore how faculty judge the "entrustability" of residents based on observing OSCE cases and then how these entrustments relate to OSCE skills performance. METHODS: In an 11-case OSCE for primary care residents (n = 25; PGY1-3), SPs rated skills in communication (information gathering, relationship development, education/counseling), assessment, patient education (case-specific), physical exam, professionalism, treatment plan, patient satisfaction and patient activation. Summary scores were calculated as%items rated well done (vs not or partly done; internal consistency > .72). Faculty observers then judged how much supervision the resident would need in actual practice to handle the case: 1-requires direct supervision, 2-requires indirect supervision, 3-ready for unsupervised practice, or 4- can supervise others. Mean entrustment rating across cases was correlated with clinical skills. RESULTS: Mean entrustment =2.46 (SD .37), falling between requires indirect supervision and ready for unsupervised practice. On average, residents were judged to need direct supervision in .40 cases (SD .65), indirect supervision in 4.76 (SD 2.03), ready for unsupervised practice in 2.92 (SD 1.80), and able to supervise others in 1.07 cases (SD 1.15) with PGY1 residents needing direct and indirect supervision in more cases than PGY2 and 3 (p = .037). Associations between OSCE performance and faculty entrustment ranged from essentially zero (communication sub-domains of information gathering and education/counseling; case-specific patient education; patient satisfaction) to negative (communication sub-domain of relationship development r = -.25, p = .16; professionalism r = -.21, p = .22) to positive (case-specific assessment, r = .35, p = .07; physical exam r = .30, p = .13; treatment plan r = .40, p = .04; patient activation r = .51, p = .008). Associations between skills performance and entrustment ratings varied by case. CONCLUSIONS: OSCEs provide a valuable opportunity for faculty to make entrustment judgments based on observing the same, complete encounter across many trainees. Entrustment judgments appear to be capturing elements of competence related to but different from SP assessments of performance, including especially "bottom line" aspects of practice such as assessment, physical exam, treatment plans and patient activation. Interestingly, we consider patient activation skills to be an "educationally sensitive patient outcome" i because both teachable and associated with patient outcomes and our results support the importance of this skill set

NEEDS AND OBJECTIVES: To address rising education costs, physician shortages, and the need for educational reform, several medical schools have developed accelerated 3-year MD programs. In 2013, NYU School of Medicine began its new 3-year MD program with guaranteed acceptance into residency upon graduation. Using the AAMC's 13 Core Entrustable Professional Activities for Entry into Residency (CEPAER) framework, we designed an immersive 4-hour simulated "Night on Call" (NOC) experience to compare performance of our first graduating cohort of fifteen 3-year MD students (3A), with third (3T) and fourth year (4T) students in the traditional 4-year MD program. SETTING AND PARTICIPANTS: 73 medical students (39 women, age 26.5 (+2.6) years; 36 '3T', 12 '3A', 25 '4T') completed an IRB-approvedNOC at our simulation center 4 weeks prior to the end of their third or final year of medical school. DESCRIPTION: We developed NOC to measure competence and entrustment across all 13 CEPAERs from the perspective of patients, nurses, and attendings. During the simulation, a medical student rotated through a series of 8 clinical coverage scenarios including: 4 standardized patient (SP) cases with varying degrees of complexity, each of which require answering a call from a standardized nurse (SN), evaluating an SP with the SN in the room, making immediate management decisions and writing a coverage note; a phone call to an experienced clinician to orally present (OP) the case; formulation of a clinical question and finding the most appropriate evidence-based medicine (EBM) answer using digital library resources; a clinical vignette (CV) to test ability to recognize a pre-entrustable peer; and a handoff (HO) of 4 cases to a peer (a senior medical student). CEPAERs assessments based on validated tools included communication, physical exam, patient education and interprofessional teamwork skills assessed by an SP and SN, and clinical reasoning based on notes, OP, EBM, CV, HO. Each rater also provided an entrustment judgment. EVALUATION: Although overall student performance improved across cases and some interesting individual performance patterns emerged, there were no significant differences across the three groups in the core competency and entrustment measures evaluated across various NOC activities. DISCUSSION/REFLECTION/LESSONS LEARNED: The 13 CEPAERs are meant to define what students should be expected to perform (without direct supervision) prior to entering residency. Our results, based on multiple rater perspectives, suggest that our cohort of 3A students is as prepared for residency as their 4T counterparts

BACKGROUND: Training residents to effectively practice value-based care is challenging. We hypothesized that residents with better communication skills would order fewer unnecessary tests and prescribe more appropriate care. We used a USP case of a patient with uncontrolled asthma to examine the relationship between value-based care and communication skills. METHODS: A 25 year-old female USP presented as a new patient to a medicine resident's clinic, reporting asthma since childhood with worsening symptoms over the past few months. At the time of the visit, she was using her albuterol inhaler multiple times daily, without any additional asthma treatment, and was unsure whether she was using it properly. Data was collected using two forms of assessment: a post-visit USP checklist and a systematic review of the corresponding clinic note to examine treatment recommendations including referrals and quality of documentation. The USP checklist measured communication, patient education, and assessment skills. Each response option included descriptive behavioral anchors and was rated as not done, partly done, or well done. Domain scores were calculated as percent items rated well done. RESULTS: 141 USP visits were made from 2009 to 2016 with a mean visit length = 88 min, SD= 28 min (range: 40 to 180 min). Almost all residents (92%) evaluated the patient's asthma with a pulmonary examination. The most common treatment prescribed was albuterol and an inhaled steroid, with or without a spacer (79%). The majority of residents (53%) did not order any additional studies; 21% ordered one study, and 26% ordered two or more studies. Study orders fell into one of three categories: gold (appropriate/recommended: PFTs, flu shot, HIV), grey (pulmonary consult, HCG), or inappropriate (TSH, A1C). Across the 141 visits, 129 studies were ordered; 46% were gold, 5% were grey, and 49% were inappropriate. The most common study ordered was a PFT (31%). 87% of single study orders were gold, but 92% of multiple orders included at least one inappropriate study. Residents who did not order any studies had significantly higher patient education and counseling skills than residents who ordered one or more studies (54% vs 34%, p = 0.00) and were more likely to explain how to correctly use an inhaler than residents who ordered one or more studies (48% vs 27%, p = 0.01). These residents also had significantly higher management and treatment skills (61% vs 39%, p = 0.00) and overall communication skills (68% vs 55%, p = 0.01). There were no significant differences between groups in medications prescribed or in quality of documentation. CONCLUSIONS: Effective communication skills may contribute to valuebased care through appropriate patient education and ordering of fewer inappropriate studies. Rigorous curricula and assessment of resident's patient education skills should be in place to help both patients and health care system achieve value-based care

Introduction Physicians often struggle with the intricacies of brain death determination and communication about end-of-life care. In an effort to remedy this situation, we introduced an educational initiative at our medical school to improve student comprehension and comfort dealing with brain death. Methods Beginning in July 2017, students at our medical school were required to attend a 90-minute brain death didactic and simulation session during their neurology clerkship. Students completed a test immediately before and after participating in the initiative. Results Of the 145 students who participated in this educational initiative between July 2016 and June 2017, 124 (86%) consented to have their data used for research purposes. Students correctly answered a median of 53% of questions (IQR 47-58%) on the pretest and 86% of questions (IQR 78-89%) on the posttest (p<0.001). Comfort with both performing a brain death evaluation and talking to a family about brain death improved significantly after this initiative (18% of students were comfortable performing a brain death evaluation before the initiative and 86% were comfortable doing so after the initiative, p<0.001; 18% were comfortable talking to a family about brain death before the initiative and 76% were comfortable doing so after the initiative, p<0.001). Conclusions Incorporation of simulation in undergraduate medical education is high-yield. At our medical school, knowledge about brain death and comfort performing a brain death exam or talking to a family about brain death was poor prior to development of this initiative, but awareness and comfort dealing with brain death improved significantly after this initiative. This initiative was clearly a success and can serve as a model for brain death education at other medical schools

OBJECTIVES: Professionalism education is a vital component of surgical training. This research attempts to determine whether an annual, year-long professionalism curriculum in a large surgical residency can effectively change professionalism attitudes. SUMMARY OF BACKGROUND DATA: The ACGME mandated 6 competencies in 2003. The competencies of Professionalism and Interpersonal/Professional Communication Skills had never been formally addressed in surgical resident education in the past. METHODS: A professionalism curriculum was developed focusing on specific resident professionalism challenges: admitting mistakes, effective communication with colleagues at all levels, delivering the news of an unexpected death, interdisciplinary challenges of working as a team, the cultural challenge of obtaining informed consent through an interpreter, and the stress of surgical practice on you and your family. These professionalism skills were then evaluated with a 6-station Objective Structured Clinical Examination (OSCE). Identical OSCE scenarios were administered to 2 cohorts of surgical residents: in 2007 (before instituting the professionalism curriculum in 2008) and again in 2014. Surgical residents were rated by trained Standardized Patients according to a behaviorally anchored professionalism criteria checklist. RESULTS: An analysis of variance was conducted of overall OSCE professionalism scores (% well done) as the dependent variable for the 2 resident cohorts (2007 vs 2014). The 2007 residents received a mean score of 38% of professionalism items "well done" (SD 9%) and the 2014 residents received a mean 59% "well done" (SD 8%). This difference is significant (F = 49.01, P < .001). CONCLUSIONS: Professionalism education has improved surgical resident understanding, awareness, and practice of professionalism in a statistically significant manner from 2007 to 2014. This documented improvement in OSCE performance reflects the value of a professionalism curriculum in the care of the patients we seek to serve.

BACKGROUND: Interprofessional collaboration (IPC) is essential for quality care. Understanding residents' level of competence is a critical first step to designing targeted curricula and workplace learning activities. In this needs assessment, we measured residents' IPC competence using specifically designed Objective Structured Clinical Exam (OSCE) cases and surveyed residents regarding training needs. METHODS: We developed three cases to capture IPC competence in the context of physician-nurse collaboration. A trained actor played the role of the nurse (Standardized Nurse - SN). The Interprofessional Education Collaborative (IPEC) framework was used to create a ten-item behaviorally anchored IPC performance checklist (scored on a three-point scale: done, partially done, well done) measuring four generic domains: values/ethics; roles/responsibilities; interprofessional communication; and teamwork. Specific skills required for each scenario were also assessed, including teamwork communication (SBAR and CUS) and patient-care-focused tasks. In addition to evaluating IPC skills, the SN assessed communication, history-taking and physical exam skills. IPC scores were computed as percent of items rated well done in each domain (Cronbach's alpha > 0.77). Analyses include item frequencies, comparison of mean domain scores, correlation between IPC and other skills, and content analysis of SN comments and resident training needs. RESULTS: One hundred and seventy-eight residents (of 199 total) completed an IPC case and results are reported for the 162 who participated in our medical education research registry. IPC domain scores were: Roles/responsibilities mean = 37 % well done (SD 37 %); Values/ethics mean = 49 % (SD 40 %); Interprofessional communication mean = 27 % (SD 36 %); Teamwork mean = 47 % (SD 29 %). IPC was not significantly correlated with other core clinical skills. SNs' comments focused on respect and IPC as a distinct skill set. Residents described needs for greater clarification of roles and more workplace-based opportunities structured to support interprofessional education/learning. CONCLUSIONS: The IPC cases and competence checklist are a practical method for conducting needs assessments and evaluating IPC training/curriculum that provides rich and actionable data at both the individual and program levels.