Faculty Bibliography

BACKGROUND: Performance-based assessment & feedback during medical training is essential for a successful transition before moving onto residency and independent clinical practice. Learners at New York University's School of Medicine (NYUSOM) participate in a routine comprehensive clinical skills examination (CCSE) that takes place at the tail end of medical school. During this exam, learners interact with standardized patients (SPs) and are rated on specific skills using a standardized checklist, measuring important clinical skills domains. NYUSOM has utilized the same assessment tool since 2005. To date, there is limited evidence on the tool's validity and ability to differentiate among students. We sought to provide evidence for it's reliability, validity, and generalizability.
METHOD(S): 1157 learners participated in the CCSE from 2011-2019 and were included in the analysis. Communication domain items assessed included patient education (3 items), relationship development (4 items), information gathering (6 items) and organization/ time management (3 items). Items were scored using a 3-point behaviorally-anchored scale (not, partly, or well done). In order to determine the degree to which the data mapped onto our theoretically-informed communication domains, we conducted a four-factor confirmatory factor analysis (CFA) allowing for factors to correlate (oblique rotation) and using means and variance adjusted weighted-least squares estimation (WLSMV) in order to account for the ordered categorical nature of the communication items. Model fit was assessed using root mean square of approximation (RMSEA) < 0.08, comparative fit index (CFI) > 0.95, and standardized root mean square error (SRMR) <0.08.
RESULT(S): The model fit the data using RMSEA (0.04), CFI (0.98), and SRMR (0.05). All factors were significantly correlated with one another (p < 0.05), with the largest correlation between patient education and organization/ time management (0.86), and information gathering (0.77). The smallest correlation was between organization/ time management and information gathering (0.66). All items (factor loadings) significantly loaded on the factors they measured. Only one item had an insignificant threshold loading between partly and well done, suggesting this part of the response scale may be hard for SPs to differentiate between students with varying ability on this item. Each factor had at least one item that had a factor loading less than 0.7.
CONCLUSION(S): The analysis suggests each item on the communication checklist significantly measures domains they were designed to measure, and that items can be summated to compute overall scores. Domains had one item with a lower loading than the rest, suggesting these items may be measuring something different. Follow up measurement modeling and profile analysis is the next logical step in determining if there is an important sub-domain that identifies a student group operating differentially. LEARNING OBJECTIVE #1: Understand clinical communication LEARNING OBJECTIVE #2: Describe communication measures

BACKGROUND: A core responsibility of medical educators is to foster a strong sense of medical professional identity (PI). Few studies specifically examine the qualities that constitute the PI of physicians recognized for exemplary professionalism. We describe those qualities based on an assessment of PI to inform educational efforts and support learners' development of PI.
METHOD(S): We used Colby and Damon's criteria for selection of moral exemplars (1992) to invite nominations of exemplary faculty physicians at NYUGSOM from faculty and trainees. Participants completed the Professional Identity Essay (PIE), a 9-question reflective writing measure based on a wellknown model of adult development that explores meaning making on PI (Bebeau & Lewis, 2004; Kegan, 1982, 1994). Two raters with extensive training and experience in adult developmental theory rated PIE responses for stage or transition phase. PI stages include independent operator, teamoriented idealist, self-defining, and self-transforming. These stages reflect increasing complexity and internalization of PI. We also gathered information on specialty, years in practice, gender, and race/ethnicity.
RESULT(S): Two hundred and twelve faculty were nominated; 35 were invited to participate (based on number of nominations, diversity of ages, backgrounds and career stage), and 21 completed scorable PIEs. They were from 13 specialties; mean career length was 21.5 years (range 6-45), and 35% were female. All but 2 were Caucasian. PIE scores ranged from 3 to 4.5 (Table 1), demonstrating differing and increasingly complex and internalized ways faculty understand their PI, and that not all nominated exemplars share a singular view of professionalism.
CONCLUSION(S): Physicians nominated as exemplars of professionalism embody a range of professional identities and professionalism world-views. Our study provides rich descriptions of multiple pathways to strengthening a physician's professionalidentities, of critical importance to faculty and physician development in a milieu of challenges to recruitment and retention of physicians. This approach can also inform educators' efforts to support PI development in learners and support the development of learning communities that foster a growth mindset. LEARNING OBJECTIVE #1: Recognize importance of strong role models for MPI. LEARNING OBJECTIVE #2: Describe the varying levels of MPI in a cohort of exemplar physicians

BACKGROUND: NYU uses the same 14-item checklist for assessing medical student communication skills across our curriculum, which includes highquality Objective Structured Clinical Skills Exams throughout the first three years of medical school: a 3-station Introductory Clinical Experience OSCE (ICE), a 3-station end-of-clinical skills OSCE (Practice of Medicine; POM); and an 8-station, high- stakes OSCE (Comprehensive Clinical Skills Exam; CCSE) after core clerkship. We describe how skills change throughout school and explore how patterns vary by cohort (class) in ways that could be explained by admissions criteria, measurement quality, and/or curriculum changes.
METHOD(S): Three domains are assessed: Info gathering (6 items), relationship development (5 items); and patient education & counseling (3 items). Checklist items use a 3-point scale (not done, partly, well done) with behavioral anchors. Internal consistency (Cronbach's alpha) exceeds .75 for all subdomains and across all years. Domains are supported by Confirmatory Factor Analysis. Mean average % well done was calculated across cases and individuals for each subdomain in an OSCE and compared over the OSCEs and between 8 classes of medical school students entering from 2009 to 2016 (graduating 2013 to 2020) (n=1569).
RESULT(S): Cohorts showed similar patterns communication skills trajectories - improvement over time. Despite changes in admissions criteria and processes, cohorts did not differ in terms of demographics, undergraduate GPA, or MCAT scores. Variability in scores decreased in all cohorts over time while communication improved. Patient education & counseling was significantly and substantially lower than other domains. In terms of cohort effects, communication scores for the entering class of 2013 at the start of medical school (ICE OSCE) were significantly higher than the previous 4. At the end of MS2, scores were similar for cohorts for info gathering and relationship development domains (and high, mean range=77-87% well done) but patient education & counseling varied: Improvement from the 1st to 3rd cohort and then decline for the last 5 cohorts. Within the CCSE (8-station pass/fail, MS3), communication scores increased steadily across entering classes, especially from cohort 4 on. These changes over time and between cohorts were mapped onto a priori descriptions of curricular, measurement and admission changes.
CONCLUSION(S): Our cohort data showed interesting and complex patterns. This study reinforces some limitations of linking curriculum to performance (e.g., no direct measures of the curriculum in terms of content, process and intensity over time, limited data on what makes cohorts different, variable measurement over time, and being unable to control for broader trends likely to influence both cohort and time effects) while also demonstrating the promise of longitudinal perspectives on the development of core competencies. LEARNING OBJECTIVE #1: Understand cohort performance in relation to curricular trends. LEARNING OBJECTIVE #2: Describe variation in performance

STATEMENT OF PROBLEM OR QUESTION (ONE SENTENCE): In order to describe quality improvement (QI) methods for health systems, we report on 10-years of using Unannounced Standardized Patient (USP) visits as the core of a program of education, training, and improvement in a system serving vulnerable patients in partnership with an academic medical center. LEARNING OBJECTIVES 1: Consider methods for supporting learning healthcare systems LEARNING OBJECTIVES 2: Identify performance data to improve care DESCRIPTION OF PROGRAM/INTERVENTION, INCLUDING ORGANIZATIONAL CONTEXT (E.G. INPATIENT VS. OUTPATIENT, PRACTICE OR COMMUNITY CHARACTERISTICS): The IOM defines a Learning Healthcare System (LHCS) as one in which science, informatics, incentives and culture are aligned for continuous improvement and innovation and where best practices are seamlessly embedded in the delivery process and new knowledge is captured as an integral by-product of the delivery experience. As essential as electronic health records are to LHCS, such data fail to capture all actionable information needed to sustain learning within complex systems. USPs are trained actors who present to clinics, incognito, to portray standardized chief complaints, histories, and characteristics. We designed and delivered USP visits to two urban, safety net clinics, focusing on assessing physician, team, and clinical micro system functioning. MEASURES OF SUCCESS (DISCUSS QUALITATIVE AND/OR QUANTITATIVEMETRICSWHICHWILL BE USEDTOEVALUATE PROGRAM/INTERVENTION): Behaviorally anchored assessments are used to assess core clinical skills (e.g., communication, information gathering, patient education, adherence to guidelines, patient centeredness, and patient activation). Team functioning assessments include professionalism and coordination. Micro system assessment focuses on safety issues like identity confirmation, hand washing, and navigation. Data from these visits has been provided to the residency, primary care teams, and to leadership and have been used to drive education, team training, and QI. FINDINGS TO DATE (IT IS NOT SUFFICIENT TO STATE FINDINGS WILL BE DISCUSSED): 1111 visits have been sent to internal medicine and primary care residents and their teams/clinics. At the resident level, needs for additional education and training in depression management, opioid prescribing, smoking cessation, and patient activation were identified and informed education. Chart reviews found substantial variation in ordering of labs and tests. At the team level, USPs uncovered needs for staff training, enhanced communication, and better processes for eliciting and documenting Social Determinants of Health (SDoH). Audit/feedback reports on provider responses to embedded SDoH combined with targeted education/resources, were associated with increased rates of eliciting and effectively responding to SDoH. In the early COVID wave, USPs tested clinic response to a potentially infectious patient. Currently, USPs are being deployed to understand variability in patients' experience of telemedicine given the rapid transformation to this modality. Finally, generalizable questions about underlying principles of medical education and quality improvement are being asked & answered using USP data to foster deeper understanding of levers for change. KEY LESSONS FOR DISSEMINATION (WHAT CAN OTHERS TAKE AWAY FOR IMPLEMENTATION TO THEIR PRACTICE OR COMMUNITY): A comprehensive USP program can provide unique insights for driving QI and innovation and help sustain a LHCS

Objectives While the need to address patients' social determinants of health (SDoH) is widely recognized, less is known about physicians' actual clinical problem-solving when it comes to SDoH. Do physicians include SDoH in their assessment strategy? Are SDoH incorporated into their diagnostic thinking and if so, do they document as part of their clinical reasoning? And do physicians directly address SDoH in their "solution" (treatment plan)? Methods We used Unannounced Standardized Patients (USPs) to assess internal medicine residents' clinical problem solving in response to a patient with asthma exacerbation and concern that her moldy apartment is contributing to symptoms - a case designed to represent a clear and direct link between a social determinant and patient health. Residents' clinical practices were assessed through a post-visit checklist and systematic chart review. Patterns of clinical problem solving were identified and then explored, in depth, through review of USP comments and history of present illness (HPI) and treatment plan documentation. Results Residents fell into three groups when it came to clinical problem-solving around a housing trigger for asthma: those who failed to ask about housing and therefore did not uncover mold as a potential trigger (neglectors - 21%; 14/68); those who asked about housing in negative ways that prevented disclosure and response (negative elicitors - 24%, 16/68); and those who elicited and explored the mold issue (full elicitors - 56%; 28/68). Of the full elicitors 53% took no further action, 26% only documented the mold; and 21% provided resources/referral. In-depth review of USP comments/explanations and residents' notes (HPI, treatment plan) revealed possible influences on clinical problem solving. Failure to ask about housing was associated with both contextual factors (rushed visit) and interpersonal skills (not fully engaging with patient) and with possible differences in attention ("known" vs. unknown/new triggers, usual symptoms vs. changes, not attending to relocation, etc.,). Use of close-ended questions often made it difficult for the patient to share mold concerns. Negative responses to sharing of housing information led to missing mold entirely or to the patient not realizing that the physician agreed with her concerns about mold. Residents who fully elicited the mold situation but did not take action seemed to either lack knowledge or feel that action on SDoH was outside their realm of responsibility. Those that took direct action to help the patient address mold appeared to be motivated by an enhanced sense of urgency. Conclusions Findings provide unique insight into residents' problem solving processes including external influences (e.g., time, distractions), the role of core communication and interpersonal skills (eliciting information, creating opportunities for patients to voice concerns, sharing clinical thinking with patients), how traditional cognitive biases operate in practice (premature closure, tunneling, and ascertainment bias), and the ways in which beliefs about expectancies and scope of practice may color clinical problem-solving strategies for addressing SDoH.

BACKGROUND: There's been a recent shift toward addressing social determinants of health (SDoH) during the clinical encounter through discussion and documentation. SDoH documentation in the problem list and through billing-related z-code use is necessary for accurate, individual patient and population level tracking andmay improve quality of care.We sought to better understand if/how providers document their patient's SDoH when elicited during a clinical visit.
METHOD(S): Unannounced Standardized Patients (USPs) were sent to two safety-net clinics to assess how medicine residents care for a new patient presenting with one of six unique chief complaints, and accompanying underlying financial, housing, and social concerns. USPs assessed resident practices after the encounter through a behaviorally anchored checklist and systematic chart review. USPs volunteered financial concerns while housing insecurity and social isolation needed to be elicited by the provider. Checklist items assessed if the USP was able to fully disclose their SDoH to the provider. Provider documentation in the electronic medical record (EMR) in one of three spaces: the history of present illness (HPI), the problem list, or through use of a social determinant-specific Zcode was examined when a USP was able to share their concerns.
RESULT(S): 384 USP visits were sent to medical residents from 2017 to 2019. USPs were able to share their financial concerns during 84% of the encounters, but were less likely to be able to share their housing or social concerns with providers (35% and 28%, respectively). Documentation in the HPI and treatment list remained low across cases (<15%) and only one Z-code was used across all visits. On an individual case level, providers addressed housing insecurity most frequently in the asthma case (discussion 65%; documentation: HPI 39%, Plan 16%) and social isolation in the fatigue case (discussion 57%; documentation: HPI 49%, Plan 2%). Providers were least likely to discuss and document SDoH for patients presenting with acute pain.
CONCLUSION(S): In clinical scenarios where SDoH concerns were elicited, residents documented SDoH in less than half of visits. Omission of SDoH not only effects clinical care but also panel management and SDoH population-level estimations. New education strategies are needed to address resident's ability to elicit and accurately document SDoH

BACKGROUND: A mature medical professional identity (PI) is a fundamental outcome of medical education (Irby and Hamstra, 2016) and medical schools across the country are developing approaches to support professional identity formation (PIF) in students. Reflective capacity, not just in the moment but as a broad skill, is key to core professional competency and may underlie PIF (Wald, 2015). Yet the relationship between reflective capacity and PIF is not well understood. Do these two concepts assess the same developmental capacity? Is reflective capacity a prerequisite for professional identity development? This pilot study is an initial attempt to explore this issue and to examine the relationship between written reflective capacity and professional identity development.
METHOD(S): As part of a professionalism curriculum medical students complete the Professional Identity Essay (PIE) at three time points: upon entrance to the school, after basic science courses, and after clinical rotations. The PIE (Bebeau and Lewis 2004), based on Kegan's developmental model (1982), requires responses to 9 prompts which elicit conceptions of the professional role. It is scored on a 5-point scale reflecting Kegan's 5 stages, with transitional stages captured by half-points. For this study, we randomly selected 20 PIE protocols from the 100 completed by the Class of 2020 after their basic science curriculum. These were scored by three raters (VM, AK, LA). Interrater reliability was established by reaching 100% agreement within one half stage on the PIE. The same raters scored the PIE protocols with the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT), following the scoring criteria (Wald 2010, Wald 2012). For both the PIE and REFLECT we averaged the three raters into a single score. A Pearson two-tailed correlation was then computed between the two scales.
RESULT(S): Completed scores on both measures were available for 19 of the 20 PIES. The range of PIE scores was 2.5-4, as would be expected of students at this point in their careers (Kalet 2018). REFLECT scores ranged 2-4. There was a statistically significant moderate positive correlation between the PIE and REFLECT (r=.628, p=.004), with REFLECT scores explaining 39% of the variance of PIE scores.
CONCLUSION(S): The correlation between PIE scores and REFLECT ratings suggests that the PIE captures and reflects some elements of learners' reflective capacity. However there remains a large component of the PIE score not explained by reflective capacity, which suggests that the PIE, as a standalone measure of PIF, demonstrates qualities beyond reflective capacity. Further investigation is warranted in order to tease out the interplay between these two concepts. Understanding the relationship between PIF and reflective capacity can inform educators in promoting a more nuanced and sophisticated PI development in students

BACKGROUND: Social determinants of health (SDoH) play a significant role in health outcomes, but little is known about care teams' attitudes about addressing SDoH. Our safety-net clinic has begun to implement SDoH screening and referral systems, but efforts to increase clinical responses to SDoH necessitates an understanding of how providers and clinical teams see their roles in responding to particular SDoH concerns.
METHOD(S): An annual survey was administered (anonymously) to clinical care teams in an urban safety-net clinic from 2017-2019, asking about ten SDoH conditions (mental health, health insurance, food, housing, transportation, finances, employment, child care, education and legal Aid). For each, respondents rated with a 4-point Likert-scale whether they agreed that health systems should address it (not at all, a little, somewhat, a great deal). They also indicated their agreement (using strongly disagree, somewhat disagree, somewhat agree, strongly agree) with two statements 1) resources are available for SDoH and 2) I can make appropriate referrals.
RESULT(S): 232 surveys were collected (103 residents, 125 faculty and staff (F/S), 5 unknown) over three years. Of note, mental health (84%) and health insurance (79%) were seen as very important for health systems to address, with other SDoH items seen as very important by fewer respondents. They reported little confidence that the health system had adequate resources (51%) and were unsure how to connect patients with services (39%). When these results were broken out by year, we found the following: In 2017 (n=77), approximately 35% of respondents thought the issues of employment, childcare, legal aid, and adult education should be addressed "a little," but in 2018 (n=81) and 2019 (n=74) respondents found the health system should be more responsible, with over 35% of respondents stating that these four issues should be addressed "somewhat" by health systems. In addition, half of respondents in 2019 felt that financial problems should be addressed "a great deal," up from 31% in 2017. Across all years, food, housing, mental health, and health insurance were seen as SDoH that should be addressed "a great deal". It is of note that respondents across all years reported limited understanding of referral methods and options available to their patients.
CONCLUSION(S): Many of the SDoH conditions were seen by respondents as outside the purview of health systems. However, over the three years, more members increased the number of SDoH conditions that should be addressed a "great deal." Responses also indicated that many of the team members do not feel prepared to deal with "unmet social needs". Additional examination of clinic SDoH coding, referral rates, resources, and team member perspectives will deepen our understanding of how we can cultivate a culture that enables team members to respond to SDoH in a way that is sensitive to their needs and patient needs

BACKGROUND: While much is known about the importance of addressing Social Determinants of Health (SDoH), less is known about how physicians elicit, respond to, and act upon their patients' SDoH information. We report on the results of a study that 1) sent Unannounced Standardized Patients (USPs) with programmed SDoH into clinics to assess whether providers uncovered, explored and acted upon the SDoH, 2) provided audit/feedback reports with educational components to clinical teams, and 3) tracked the impact of that intervention on provider response to SDoH.
METHOD(S): Highly trained USPs (secret shoppers) portrayed six scenarios (fatigue, asthma, Hepatitis B concern, shoulder pain, back pain, well-visit), each with specific housing (overcrowding, late rent, and mold) and social isolation (shyness, recent break up, and anxiety) concerns that they shared if asked broadly about. USPs assessed team and provider SDoH practices (eliciting, acknowledging/exploring, and providing resources and/or referrals). 383 USP visits were made to residents in 5 primary care teams in 2 urban, safety- net clinics. 123 visits were fielded during baseline period (Feb 2017-Jan 2018); 185 visits during intervention period (Jan 2018-Mar 2019) throughout which quarterly audit/feedback reports of the teams' response to the USPs' SDoH and targeted education on SDoH were distributed; and 75 follow-up phase visits were fielded (Apr-Dec 2019). Analyses compared rates of eliciting and responding to SDoH across the 3 periods (chi- square, z-scores). One team, by design, did not receive the intervention and serves as a comparison group.
RESULT(S): Among the intervention teams, the rate of eliciting the housing SDoH increased from 46% at baseline to 59% during the intervention period (p=.045) and also increased, but not significantly, for the social issue (40% to 52%, p=.077). There was a significant increase from baseline to intervention in providing resources/referrals for housing (from 7% to 24%, p=.001) and for social isolation (from 13% to 24%, p=.042) (mostly resources, very few referrals were made). The comparison team's rates followed a different pattern: eliciting the housing issue and the social isolation issue decreased from baseline to the intervention period (housing: 61% to 45%; social isolation: 39% to 33% of visits) and the rate of providing resources/referrals stayed steady at 13% for both. In the cases where SDoH were most clinically relevant, baseline rates of identifying the SDoH were high (>70%) but rates of acting on the SDoH increased significantly from baseline to intervention. Increases seen in the intervention period were not sustained in the follow-up period.
CONCLUSION(S): Giving providers SDoH data along with targeted education was associated with increased but unsustained rates of eliciting and responding to housing and social issues. The USP methodology was an effective means of presenting controlled SDoH and providing audit/feedback data. Ongoing education and feedback may be needed