Viewing Readiness-for-Residency through Binoculars: Mapping Competency-Based Assessments to the AAMC's 13 Core Entrustable Professional Activities (EPAs)
Does a measure of Medical Professional Identity Formation predict communication skills performance?
OBJECTIVE:To validate an approach to measuring professional identity formation (PIF), we explore if the Professional Identity Essay (PIE), a stage score measure of medical professional identity (PI), predicts clinical communication skills. METHODS:Students completed the PIE during medical school orientation and a 3-case Objective Structured Clinical Exam (OSCE) where standardized patients reliably assessed communication skills in 5 domains. Using mediation analyses, relationships between PIE stage scores and communication skills were explored. RESULTS:For the 351 (89%) consenting students, controlling for individual characteristics, there were increases in patient counseling (6.5%, p<0.01), information gathering (4.3%, pâ€¯=â€¯0.01), organization and management (4.1%, pâ€¯=â€¯0.02), patient assessment (3.6%, pâ€¯=â€¯0.04), and relationship development (3.5%, pâ€¯=â€¯0.03) skills for every half stage increase in PIE score. The communication skills of lower socio-economic status (SES) students are indirectly impacted by their slightly higher PIE stage scores. CONCLUSION/CONCLUSIONS:Higher PIE stage scores are associated with higher communication skills and lower SES. PRACTICE IMPLICATIONS/CONCLUSIONS:PIE predicts critical clinical skills and identifies how SES and other characteristics indirectly impact future clinical performance, providing validity evidence for using PIE as a tool in longitudinal formative academic coaching, program and curriculum evaluation, and research.
Changing hats: Lessons learned integrating coaching into UME and GME [Meeting Abstract]
BACKGROUND: The transition from medical school to residency is characterized by an abrupt transition of learning needs and goals. Coaching is a promising intervention to support individual learning and growth trajectories of learners. It is uncommon for medical school faculty to have undergone training as coaches. We explored our faculty's perceptions and skills after instituting a new coaching program.
METHOD(S): Faculty advisors (N=12) and GME (N=16) participated in a coaching development program and in community of practice meetings where challenging coaching scenarios were shared. GME faculty also participated in a Group Objective Structured Clinical Exam (GOSCE) to practice and receive feedback on their skills. Peer-faculty observers and resident raters used behaviorally grounded checklists to assess faculty performance. We conducted 2 focus groups: 1) UME advisors engaged in longitudinal coaching (n=9) and 2) GME faculty participating in the coaching development program (n=8) to better understand how faculty make sense of and put into practice these new coaching roles and skills.
RESULT(S): Simple thematic coding showed that both groups emphasized the blurring of the many roles they serve when interacting with trainees and struggled with recognizing both which hat to wear (role to adopt) and which skills to call upon in specific situations. UME advisors who have dedicated advising/coaching roles reported assuming multiple roles at different times with their same students. Many of the GME coaches serve as Associate Program Directors, and described adopting a coaching frame of reference (mentality) and requiring external reinforcement for coaching skills. Some reported realizing after the fact that coaching would have been a valuable approach. Faculty newer to their role felt more successful in engaging in coaching mindset and coaching. Faculty were curious about how trainees would feel about this approach and anticipated that some would appreciate this more than others. 12 faculty participated in a three station Coaching GOSCE. Both resident raters and faculty peer raters suggested faculty coaches were able to establish trust and engage in authentic listening. Coaches negotiated the tension between empathetic listening with supporting goal-setting. Residents provided slightly lower ratings than peer observers on coaches' ability to ask questions and assume a coachee- focused agenda.
CONCLUSION(S): Medical educators may benefit from obtaining coaching skills, but deliberate training in how these skills complement, and differ, from existing skills requires both didactic and experiential learning. Cultivating a community of practice and offering opportunities for deliberate practice, observation and feedback is essential for medical educators to achieve mastery as coaches. LEARNING OBJECTIVE #1: Identify and perform appropriate learning activities to guide personal and professional development (PBL) LEARNING OBJECTIVE #2: Understand and apply core longitudinal coaching skills (Professionalism)
Describing faculty exemplars of medical professionalism [Meeting Abstract]
BACKGROUND: Internalizing a strong medical professional identity (PI) is a critical part of medical education. Recent studies of medical students have documented that students' PI, measured by the Professional Identity Essay (PIE), a reflective writing assessment of PI based on Kegan's theory of adult development and Bebeau's developmental model of PI, vary and are impacted by education. Little is known about the PI of exemplary professional physicians. We sought to: 1) describe the PI of physicians who exemplify the highest principles of the medical profession, and 2) evaluate NYU faculty identified as professional exemplars by peers to provide data and demonstrate clear role models for learners METHODS: We elicited nominations for professional exemplar physicians from NYU faculty, chief residents, and 4th-year students, using the definition of professionalism developed by Colby and Damon (1992). Participants were recruited after receiving at least 3 nominations; select participants who received 1 or 2 nominations were also recruited to diversify the participants in terms of specialty, years of practice, gender and race.We also used snowball techniques to get nominations fromstudy participants. After consenting, faculty received the 11-question PIE. We analyzed demographic data of nominated faculty and completed a content analysis of the PIE.
RESULT(S): 206 individual faculty were nominated at least one time by 70 community members. 32 individuals were recruited to the study; to date 22 have completed the PIE. The 206 nominees/22 participants represent: 34/12 specialties, average years in practice 17.6/23.8, range of years in practice 62 for nominees/44 for participants. We identified 3 primary themes through the content analysis: (1) Response to Expectations, "Everything. The profession demands everything.As much as this profession takes fromme, it is dwarfed by what I have received in return." (2) Response to Failure: "I fail to live up to expectations every day. Some days thismotivatesme, other days I disappoint myself." (3) Learning from Others: "I view teaching as integral to medical professionalism." There was a range of developmental levels in the responses with some focusing more on external rather than internal motivations: "I can say that the [malpractice] process for me was very threatening, emotionally consuming and had the potential to alter professional behavior in the wrong way."
CONCLUSION(S): Nominated faculty represented a diverse group with respect to PI. Many participants demonstrated great professionalism and a sense of internal PI in responses to the PIE questions, while others focused onmore externalmotivations to drive their professional behaviors. Further analysis is needed to define the qualities of a true exemplary professional. The range of responses of the exemplars can both serve as role models for learners and provide multiple pathways for learners and faculty to strengthen their own professional identities
A Culture of Safety From Day 1: An Institutional Patient Safety Initiative to Support Incoming Interns
SIMULATED FIRST NIGHT-ONCALL (FNOC): ESTABLISHING COMMUNITY AND A CULTURE OF PATIENT SAFETY FOR INCOMING INTERNS [Meeting Abstract]
TRANSITIONING TO RESIDENCY IN THE ERA OF EPAS: MAPPING CLINICAL SIMULATION MEASURES TO THE 13 CORE EPAS' "ENTRUSTABLE BEHAVIORS" [Meeting Abstract]
Measuring professional identity formation early in medical school: Validity evidence [Meeting Abstract]
BACKGROUND: An evidence-based approach to understand ethical medical professional identity formation (PIF) is needed. Scored by an expert, the Professional Identity Essay (PIE) produces a developmental stage (8 potential levels) based on a constructivist-developmental methodology developed in dental, law students and military officer trainees. We assessed the validity and utility of measuring baseline PIF in a professionalism curriculum and hypothesized that PIE stage would correlate with undergraduate humanities majors, subscores on admissions Multiple Mini Interviews (MMI), and DIT2 score, but would not correlate with gender, SES, URM status, age, MCAT scores, or GPA. METHODS: During medical school orientation, all 132 entering medical students completed 1) the Professional Identity Essay (PIE) and 2) the Defining Issues Test (DIT2), a validated measure of moral reasoning. A trained expert (VM) scored narrative responses to seven prompts (Inter-rater ICC .83, 95% CI [.57 - .96], intra-rater ICC .85, 95% CI [.50 - .93]). The DIT2 N2 score reflects the proportion of the time students used universal ethical principles to justify a response to six moral dilemma cases. Students' reflections following a debriefing were content-analyzed. Admissions data was obtained for consenting students. RESULTS: 129 students consented (57% women, mean age 23 (range 18-34, SD 1.7). Distribution of PIF stage scores was along 4 stages (2-3: 16 (12%), 3: 59 (45%), 3-4: 49 (37%) 4: 6 (4%)). DIT2 scores indicated a strong preference for post-conventional moral reasoning (N2) relative to moral justifications based on personal interest or maintaining norms, with a mean DIT2 N2 score of 54% (range 9.7%-76.5%). PIF stage as measured by the PIE was correlated with DIT N2, rho = .19 (p < .05), MMI-Conflict Resolution, rho = .26 (p < .05) and MMI-Team Work, rho = .22 (p < .05), but no other admission variable. DIT N2 score and MMI-Responsibility were correlated rho = .18 (p < .05). Content analysis of 130 students' reflective writing on PIE stage and DIT score report revealed mostly positive reactions to the PIF curriculum (117/130, 90%). Students with mixed negative reactions (28/130, 21%) reported confusion, concern about future training, or dissatisfaction with the gap between their self-perception and the results. Students who demonstrated an analytic approach to evaluating their feedback tended not to report negative reactions, even when they were surprised by the feedback. CONCLUSIONS: A developmental evidence grounded measure of PIF is a feasible and acceptable part of a medical professionalism curriculum for entering medical students, distributes similarly to groups in other professional training programs and correlates with a validated measure of moral reasoning and admissions MMI stations that assess relevant constructs. This approach is acceptable and intriguing to our students and may be useful as an approach to competency-based professionalism curricula
Are accelerated 3-year md pathway students prepared for day one of internship? [Meeting Abstract]
NEEDS AND OBJECTIVES: To address rising education costs, physician shortages, and the need for educational reform, several medical schools have developed accelerated 3-year MD programs. In 2013, NYU School of Medicine began its new 3-year MD program with guaranteed acceptance into residency upon graduation. Using the AAMC's 13 Core Entrustable Professional Activities for Entry into Residency (CEPAER) framework, we designed an immersive 4-hour simulated "Night on Call" (NOC) experience to compare performance of our first graduating cohort of fifteen 3-year MD students (3A), with third (3T) and fourth year (4T) students in the traditional 4-year MD program. SETTING AND PARTICIPANTS: 73 medical students (39 women, age 26.5 (+2.6) years; 36 '3T', 12 '3A', 25 '4T') completed an IRB-approvedNOC at our simulation center 4 weeks prior to the end of their third or final year of medical school. DESCRIPTION: We developed NOC to measure competence and entrustment across all 13 CEPAERs from the perspective of patients, nurses, and attendings. During the simulation, a medical student rotated through a series of 8 clinical coverage scenarios including: 4 standardized patient (SP) cases with varying degrees of complexity, each of which require answering a call from a standardized nurse (SN), evaluating an SP with the SN in the room, making immediate management decisions and writing a coverage note; a phone call to an experienced clinician to orally present (OP) the case; formulation of a clinical question and finding the most appropriate evidence-based medicine (EBM) answer using digital library resources; a clinical vignette (CV) to test ability to recognize a pre-entrustable peer; and a handoff (HO) of 4 cases to a peer (a senior medical student). CEPAERs assessments based on validated tools included communication, physical exam, patient education and interprofessional teamwork skills assessed by an SP and SN, and clinical reasoning based on notes, OP, EBM, CV, HO. Each rater also provided an entrustment judgment. EVALUATION: Although overall student performance improved across cases and some interesting individual performance patterns emerged, there were no significant differences across the three groups in the core competency and entrustment measures evaluated across various NOC activities. DISCUSSION/REFLECTION/LESSONS LEARNED: The 13 CEPAERs are meant to define what students should be expected to perform (without direct supervision) prior to entering residency. Our results, based on multiple rater perspectives, suggest that our cohort of 3A students is as prepared for residency as their 4T counterparts
Measuring professional identity formation early in medical school
AIM: To assess the feasibility and utility of measuring baseline professional identity formation (PIF) in a theory-based professionalism curriculum for early medical students. METHODS: All 132 entering students completed the professional identity essay (PIE) and the defining issues test (DIT2). Students received score reports with individualized narrative feedback and wrote a structured reflection after a large-group session in which the PIF construct was reviewed. Analysis of PIEs resulted in assignment of a full or transitional PIF stage (1-5). The DIT2 score reflects the proportion of the time students used universal ethical principles to justify a response to 6 moral dilemma cases. Students' reflections were content analyzed. RESULTS: PIF scores were distributed across stage 2/3, stage 3, stage 3/4, and stage 4. No student scores were in stages 1, 2, 4/5, or 5. The mean DIT2 score was 53% (range 9.7?76.5%); the correlation between PIF stage and DIT score was rho = 0.18 (p = 0.03). Students who took an analytic approach to the data and demonstrated both awareness that they are novices and anticipation of continued PIF tended to respond more positively to the feedback. CONCLUSIONS: These PIF scores distributed similarly to novice students in other professions. Developmental-theory based PIF and moral reasoning measures are related. Students reflected on these measures in meaningful ways suggesting utility of measuring PIF scores in medical education.