Faculty Bibliography

BACKGROUND: NYU uses the same 14-item checklist for assessing medical student communication skills across our curriculum, which includes highquality Objective Structured Clinical Skills Exams throughout the first three years of medical school: a 3-station Introductory Clinical Experience OSCE (ICE), a 3-station end-of-clinical skills OSCE (Practice of Medicine; POM); and an 8-station, high- stakes OSCE (Comprehensive Clinical Skills Exam; CCSE) after core clerkship. We describe how skills change throughout school and explore how patterns vary by cohort (class) in ways that could be explained by admissions criteria, measurement quality, and/or curriculum changes.
METHOD(S): Three domains are assessed: Info gathering (6 items), relationship development (5 items); and patient education & counseling (3 items). Checklist items use a 3-point scale (not done, partly, well done) with behavioral anchors. Internal consistency (Cronbach's alpha) exceeds .75 for all subdomains and across all years. Domains are supported by Confirmatory Factor Analysis. Mean average % well done was calculated across cases and individuals for each subdomain in an OSCE and compared over the OSCEs and between 8 classes of medical school students entering from 2009 to 2016 (graduating 2013 to 2020) (n=1569).
RESULT(S): Cohorts showed similar patterns communication skills trajectories - improvement over time. Despite changes in admissions criteria and processes, cohorts did not differ in terms of demographics, undergraduate GPA, or MCAT scores. Variability in scores decreased in all cohorts over time while communication improved. Patient education & counseling was significantly and substantially lower than other domains. In terms of cohort effects, communication scores for the entering class of 2013 at the start of medical school (ICE OSCE) were significantly higher than the previous 4. At the end of MS2, scores were similar for cohorts for info gathering and relationship development domains (and high, mean range=77-87% well done) but patient education & counseling varied: Improvement from the 1st to 3rd cohort and then decline for the last 5 cohorts. Within the CCSE (8-station pass/fail, MS3), communication scores increased steadily across entering classes, especially from cohort 4 on. These changes over time and between cohorts were mapped onto a priori descriptions of curricular, measurement and admission changes.
CONCLUSION(S): Our cohort data showed interesting and complex patterns. This study reinforces some limitations of linking curriculum to performance (e.g., no direct measures of the curriculum in terms of content, process and intensity over time, limited data on what makes cohorts different, variable measurement over time, and being unable to control for broader trends likely to influence both cohort and time effects) while also demonstrating the promise of longitudinal perspectives on the development of core competencies. LEARNING OBJECTIVE #1: Understand cohort performance in relation to curricular trends. LEARNING OBJECTIVE #2: Describe variation in performance

BACKGROUND: Performance-based assessment & feedback during medical training is essential for a successful transition before moving onto residency and independent clinical practice. Learners at New York University's School of Medicine (NYUSOM) participate in a routine comprehensive clinical skills examination (CCSE) that takes place at the tail end of medical school. During this exam, learners interact with standardized patients (SPs) and are rated on specific skills using a standardized checklist, measuring important clinical skills domains. NYUSOM has utilized the same assessment tool since 2005. To date, there is limited evidence on the tool's validity and ability to differentiate among students. We sought to provide evidence for it's reliability, validity, and generalizability.
METHOD(S): 1157 learners participated in the CCSE from 2011-2019 and were included in the analysis. Communication domain items assessed included patient education (3 items), relationship development (4 items), information gathering (6 items) and organization/ time management (3 items). Items were scored using a 3-point behaviorally-anchored scale (not, partly, or well done). In order to determine the degree to which the data mapped onto our theoretically-informed communication domains, we conducted a four-factor confirmatory factor analysis (CFA) allowing for factors to correlate (oblique rotation) and using means and variance adjusted weighted-least squares estimation (WLSMV) in order to account for the ordered categorical nature of the communication items. Model fit was assessed using root mean square of approximation (RMSEA) < 0.08, comparative fit index (CFI) > 0.95, and standardized root mean square error (SRMR) <0.08.
RESULT(S): The model fit the data using RMSEA (0.04), CFI (0.98), and SRMR (0.05). All factors were significantly correlated with one another (p < 0.05), with the largest correlation between patient education and organization/ time management (0.86), and information gathering (0.77). The smallest correlation was between organization/ time management and information gathering (0.66). All items (factor loadings) significantly loaded on the factors they measured. Only one item had an insignificant threshold loading between partly and well done, suggesting this part of the response scale may be hard for SPs to differentiate between students with varying ability on this item. Each factor had at least one item that had a factor loading less than 0.7.
CONCLUSION(S): The analysis suggests each item on the communication checklist significantly measures domains they were designed to measure, and that items can be summated to compute overall scores. Domains had one item with a lower loading than the rest, suggesting these items may be measuring something different. Follow up measurement modeling and profile analysis is the next logical step in determining if there is an important sub-domain that identifies a student group operating differentially. LEARNING OBJECTIVE #1: Understand clinical communication LEARNING OBJECTIVE #2: Describe communication measures

BACKGROUND: A core responsibility of medical educators is to foster a strong sense of medical professional identity (PI). Few studies specifically examine the qualities that constitute the PI of physicians recognized for exemplary professionalism. We describe those qualities based on an assessment of PI to inform educational efforts and support learners' development of PI.
METHOD(S): We used Colby and Damon's criteria for selection of moral exemplars (1992) to invite nominations of exemplary faculty physicians at NYUGSOM from faculty and trainees. Participants completed the Professional Identity Essay (PIE), a 9-question reflective writing measure based on a wellknown model of adult development that explores meaning making on PI (Bebeau & Lewis, 2004; Kegan, 1982, 1994). Two raters with extensive training and experience in adult developmental theory rated PIE responses for stage or transition phase. PI stages include independent operator, teamoriented idealist, self-defining, and self-transforming. These stages reflect increasing complexity and internalization of PI. We also gathered information on specialty, years in practice, gender, and race/ethnicity.
RESULT(S): Two hundred and twelve faculty were nominated; 35 were invited to participate (based on number of nominations, diversity of ages, backgrounds and career stage), and 21 completed scorable PIEs. They were from 13 specialties; mean career length was 21.5 years (range 6-45), and 35% were female. All but 2 were Caucasian. PIE scores ranged from 3 to 4.5 (Table 1), demonstrating differing and increasingly complex and internalized ways faculty understand their PI, and that not all nominated exemplars share a singular view of professionalism.
CONCLUSION(S): Physicians nominated as exemplars of professionalism embody a range of professional identities and professionalism world-views. Our study provides rich descriptions of multiple pathways to strengthening a physician's professionalidentities, of critical importance to faculty and physician development in a milieu of challenges to recruitment and retention of physicians. This approach can also inform educators' efforts to support PI development in learners and support the development of learning communities that foster a growth mindset. LEARNING OBJECTIVE #1: Recognize importance of strong role models for MPI. LEARNING OBJECTIVE #2: Describe the varying levels of MPI in a cohort of exemplar physicians

OBJECTIVE:To validate an approach to measuring professional identity formation (PIF), we explore if the Professional Identity Essay (PIE), a stage score measure of medical professional identity (PI), predicts clinical communication skills. METHODS:Students completed the PIE during medical school orientation and a 3-case Objective Structured Clinical Exam (OSCE) where standardized patients reliably assessed communication skills in 5 domains. Using mediation analyses, relationships between PIE stage scores and communication skills were explored. RESULTS:For the 351 (89%) consenting students, controlling for individual characteristics, there were increases in patient counseling (6.5%, p<0.01), information gathering (4.3%, p = 0.01), organization and management (4.1%, p = 0.02), patient assessment (3.6%, p = 0.04), and relationship development (3.5%, p = 0.03) skills for every half stage increase in PIE score. The communication skills of lower socio-economic status (SES) students are indirectly impacted by their slightly higher PIE stage scores. CONCLUSION/CONCLUSIONS:Higher PIE stage scores are associated with higher communication skills and lower SES. PRACTICE IMPLICATIONS/CONCLUSIONS:PIE predicts critical clinical skills and identifies how SES and other characteristics indirectly impact future clinical performance, providing validity evidence for using PIE as a tool in longitudinal formative academic coaching, program and curriculum evaluation, and research.

BACKGROUND: While much is known about the importance of addressing Social Determinants of Health, less is known about how members of the care team respond to patient-volunteered SDoH - especially when the determinant is related to financial insecurity. With increasing calls for universal screening for SDoH - what do teams do when a patient shares a financial concern? We report on the use of Unannounced Standardized Patients (USP) to assess how primary care teams respond to volunteered information about financial insecurity and whether an audit/ feedback intervention (with targeted education included) improved that response.
METHOD(S): Highly trained USPs (secret shoppers) portrayed six common scenarios (fatigue, asthma, Hepatitis B concern, shoulder pain, back pain, well visit). USPs volunteered a financial concern (fear of losing job, challenges with financially supporting parent, trouble meeting rent) to the medical assistant (MA) and then again to their provider and assessed the response of both the MA (did they acknowledge and/ or forward the information to the provider?) and the provider (did they acknowledge/ explore and/or provide resources/referrals?). A total of 383 USP visits were delivered to 5 care teams in 2 safety-net clinics. Providers were medicine residents. 123 visits were fielded during the baseline period (Feb 2017-Jan 2018); 185 visits during the intervention period (Jan 2018-Mar 2019) throughout which quarterly audit/feedback reports of the teams' response to the USPs' SDoH and targeted education on SDoH were distributed. 75 follow-up phase visits were fielded (Apr- Dec 2019). Analyses compared rates of MA and provider response to the volunteered financial insecurity issue across the 3 periods (chi-square, z-scores).
RESULT(S): The baseline rate of responding in some way to the volunteered information was high for both the MA (86% acknowledged) and the providers (100% responded). These overall rates of response did not change substantially or significantly across the three time periods (MA: Intervention period = 87%, Follow- Up period=90%; Provider: Intervention period=98%; Follow-Up period=98%). Rates of acting upon the volunteered information also remained quite consistent across the time periods: from 29 to 35% of MA forwarded the information to the provider across the 3 time periods and from 22 to 28% of providers in each intervention period gave the patient resources or a referral (mostly resources).
CONCLUSION(S): Our findings highlight the importance of patients directly telling team members about a financial concern. Future research should explore whether screening tools are effective in instigating a response. Audit/feedback reports with targeted educational components did not appear to influence the teams' response unlike what we found for housing and social concerns that had to be elicited. Whether this is due to differences in volunteered vs. elicited SDoH or to the nature of the SDoH (financial vs housing/social) warrants further investigation

BACKGROUND: Internalizing a strong medical professional identity (PI) is a critical part of medical education. Recent studies of medical students have documented that students' PI, measured by the Professional Identity Essay (PIE), a reflective writing assessment of PI based on Kegan's theory of adult development and Bebeau's developmental model of PI, vary and are impacted by education. Little is known about the PI of exemplary professional physicians. We sought to: 1) describe the PI of physicians who exemplify the highest principles of the medical profession, and 2) evaluate NYU faculty identified as professional exemplars by peers to provide data and demonstrate clear role models for learners METHODS: We elicited nominations for professional exemplar physicians from NYU faculty, chief residents, and 4th-year students, using the definition of professionalism developed by Colby and Damon (1992). Participants were recruited after receiving at least 3 nominations; select participants who received 1 or 2 nominations were also recruited to diversify the participants in terms of specialty, years of practice, gender and race.We also used snowball techniques to get nominations fromstudy participants. After consenting, faculty received the 11-question PIE. We analyzed demographic data of nominated faculty and completed a content analysis of the PIE.
RESULT(S): 206 individual faculty were nominated at least one time by 70 community members. 32 individuals were recruited to the study; to date 22 have completed the PIE. The 206 nominees/22 participants represent: 34/12 specialties, average years in practice 17.6/23.8, range of years in practice 62 for nominees/44 for participants. We identified 3 primary themes through the content analysis: (1) Response to Expectations, "Everything. The profession demands everything.As much as this profession takes fromme, it is dwarfed by what I have received in return." (2) Response to Failure: "I fail to live up to expectations every day. Some days thismotivatesme, other days I disappoint myself." (3) Learning from Others: "I view teaching as integral to medical professionalism." There was a range of developmental levels in the responses with some focusing more on external rather than internal motivations: "I can say that the [malpractice] process for me was very threatening, emotionally consuming and had the potential to alter professional behavior in the wrong way."
CONCLUSION(S): Nominated faculty represented a diverse group with respect to PI. Many participants demonstrated great professionalism and a sense of internal PI in responses to the PIE questions, while others focused onmore externalmotivations to drive their professional behaviors. Further analysis is needed to define the qualities of a true exemplary professional. The range of responses of the exemplars can both serve as role models for learners and provide multiple pathways for learners and faculty to strengthen their own professional identities

BACKGROUND: Unannounced Standardized Patients (USPs) provide opportunity to measure residents' clinical skills in actual practice. USPs, or secret shoppers, are trained to ensure accurate case portrayal across encounters, making them optimal for tracking changes in skills longitudinally. At present, little is known about how residents handle USP visits with repeat cases. This study examines variation in resident communication and global domain scores when visited by the sameUSPcase at two separate time points during residency training.
METHOD(S): Primary care residents (n=46) were assessed twice by one of six standardized cases (asthma, fatigue, Hepatitis B concern, back pain, shoulder pain, or well visit) during the course of their residency, typically during their first and third training year. Upon visit completion, residents were rated using a behaviorally-anchored checklist. Communication domains assessed included info gathering (4 items), relationship development (5 items), and patient education (4 items). Other domains included patient activation (4 items) and satisfaction (4 items). Responses were scored as not done, partly done, or well done. Summary scores (mean % well done) were calculated by domain. All cases were combined to create composite scores, due to small sample sizes per case. First and second visit domain scores were compared using a t-test. Finally, we grouped high performers (80% or higher on communication scores during their first visit) because this measure demonstrated competency.
RESULT(S): With cases combined, there were no significant differences based on time of assessment and changes in score between first and second visit were small. 14/46 (30%) learners who performed well on composite overall communication scores (80% or higher) during their first visit outperformed poorer communicators in patient satisfaction (93% vs 61%, P<.001) and activation (48% vs 18%, P<.001). In subsequent visits, these high performers performed at a similar level to their fellow residents, with no significant differences noted. Further, when looking at individual trajectories, individual learner scores in the communication domain increased between visits for 21 learners (46%), decreased for 19 (41%), and stayed same for 7 (15%).
CONCLUSION(S): Results suggest that a learning curve occurs between assessments during the first year in residency and subsequent assessments. This could be due to an increased capacity to engage with a patient occurring training progression, or due to a better understanding of addressing common chief complaints presented with our USP cases. Understanding causes of individual-level score decreases will enable tailoring of educational interventions suitable for specific learner trajectories, as will a deeper dive into the impact of the clinical microsystem on performance. We predict a more nuanced understanding of these mediating factors through our plan of increasing our repeat visit sample size

BACKGROUND: There's been a recent shift toward addressing social determinants of health (SDoH) during the clinical encounter through discussion and documentation. SDoH documentation in the problem list and through billing-related z-code use is necessary for accurate, individual patient and population level tracking andmay improve quality of care.We sought to better understand if/how providers document their patient's SDoH when elicited during a clinical visit.
METHOD(S): Unannounced Standardized Patients (USPs) were sent to two safety-net clinics to assess how medicine residents care for a new patient presenting with one of six unique chief complaints, and accompanying underlying financial, housing, and social concerns. USPs assessed resident practices after the encounter through a behaviorally anchored checklist and systematic chart review. USPs volunteered financial concerns while housing insecurity and social isolation needed to be elicited by the provider. Checklist items assessed if the USP was able to fully disclose their SDoH to the provider. Provider documentation in the electronic medical record (EMR) in one of three spaces: the history of present illness (HPI), the problem list, or through use of a social determinant-specific Zcode was examined when a USP was able to share their concerns.
RESULT(S): 384 USP visits were sent to medical residents from 2017 to 2019. USPs were able to share their financial concerns during 84% of the encounters, but were less likely to be able to share their housing or social concerns with providers (35% and 28%, respectively). Documentation in the HPI and treatment list remained low across cases (<15%) and only one Z-code was used across all visits. On an individual case level, providers addressed housing insecurity most frequently in the asthma case (discussion 65%; documentation: HPI 39%, Plan 16%) and social isolation in the fatigue case (discussion 57%; documentation: HPI 49%, Plan 2%). Providers were least likely to discuss and document SDoH for patients presenting with acute pain.
CONCLUSION(S): In clinical scenarios where SDoH concerns were elicited, residents documented SDoH in less than half of visits. Omission of SDoH not only effects clinical care but also panel management and SDoH population-level estimations. New education strategies are needed to address resident's ability to elicit and accurately document SDoH

BACKGROUND: A mature medical professional identity (PI) is a fundamental outcome of medical education (Irby and Hamstra, 2016) and medical schools across the country are developing approaches to support professional identity formation (PIF) in students. Reflective capacity, not just in the moment but as a broad skill, is key to core professional competency and may underlie PIF (Wald, 2015). Yet the relationship between reflective capacity and PIF is not well understood. Do these two concepts assess the same developmental capacity? Is reflective capacity a prerequisite for professional identity development? This pilot study is an initial attempt to explore this issue and to examine the relationship between written reflective capacity and professional identity development.
METHOD(S): As part of a professionalism curriculum medical students complete the Professional Identity Essay (PIE) at three time points: upon entrance to the school, after basic science courses, and after clinical rotations. The PIE (Bebeau and Lewis 2004), based on Kegan's developmental model (1982), requires responses to 9 prompts which elicit conceptions of the professional role. It is scored on a 5-point scale reflecting Kegan's 5 stages, with transitional stages captured by half-points. For this study, we randomly selected 20 PIE protocols from the 100 completed by the Class of 2020 after their basic science curriculum. These were scored by three raters (VM, AK, LA). Interrater reliability was established by reaching 100% agreement within one half stage on the PIE. The same raters scored the PIE protocols with the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT), following the scoring criteria (Wald 2010, Wald 2012). For both the PIE and REFLECT we averaged the three raters into a single score. A Pearson two-tailed correlation was then computed between the two scales.
RESULT(S): Completed scores on both measures were available for 19 of the 20 PIES. The range of PIE scores was 2.5-4, as would be expected of students at this point in their careers (Kalet 2018). REFLECT scores ranged 2-4. There was a statistically significant moderate positive correlation between the PIE and REFLECT (r=.628, p=.004), with REFLECT scores explaining 39% of the variance of PIE scores.
CONCLUSION(S): The correlation between PIE scores and REFLECT ratings suggests that the PIE captures and reflects some elements of learners' reflective capacity. However there remains a large component of the PIE score not explained by reflective capacity, which suggests that the PIE, as a standalone measure of PIF, demonstrates qualities beyond reflective capacity. Further investigation is warranted in order to tease out the interplay between these two concepts. Understanding the relationship between PIF and reflective capacity can inform educators in promoting a more nuanced and sophisticated PI development in students