Faculty Bibliography

Introduction: The objective structured clinical examination (OSCE) is a common educational strategy to assess interpersonal skills and knowledge gaps, and we have previously shown its benefits in inflammatory bowel disease (IBD) education. As a result of the COVID-19 pandemic, there has been explosive growth in telehealth. Currently, we lack methods to teach and evaluate trainees' related skillset and no telehealth-specific milestones exist. We assessed the telehealth proficiency of gastroenterology (GI) fellows as part of an annual IBD OSCE over Zoom (Z-OSCE).
Method(s): Seven GI fellows from four programs participated in Z-OSCE featuring four clinical scenarios. We used previously validated OSCE checklists to assess the fellows' performance in IBDspecific cases. Telehealth communication skills were also assessed. One of the scenarios required the trainees to conduct a televisit focusing on preventative care for a Crohn's disease patient based on the ACG guideline. Checklists were scored on a 3-point Likert scale by the Standardized Patient (SP).
Result(s): The telehealth behavioral anchored checklist items included: maintaining proper computer etiquette, use of non-verbal communication and positioning to optimize the encounter, optimized technical aspects, and exhibiting comfort and confidence with the virtual platform. In these domains 5 or 6 (out of 7) fellows received 3 points. In contrast less than half of the fellows asked questions of the SP to make sure they understood or acknowledged emotion, and 3 out 7 fellows did not make appropriate eye contact. Despite high performance in the telehealth aspects of the encounter, the SP did not fully recommend fellows for their communication skills without reservations to friends/ family, with all fellows receiving 2 points.
Conclusion(s): Telehealth has quickly become a fixture of our profession, and merits educational opportunities and assessment to improve clinicians' competency, as it will likely outlast the COVID- 19 pandemic. To our knowledge, this is the first OSCE designed to assess telehealth performance in the delivery of IBD care. Z-OSCEs can play a unique role in simulating realistic telehealth visits and this pilot program helps us identify future educational needs. Assessment of fellows' performance during this virtual program can provide an opportunity for learning, growth and reflection as well as prepare trainees for future patient encounters

Introduction: The objective structured clinical examination (OSCE) has been shown to not only assess but also improve the performance of trainees. Our group has previously demonstrated the benefits of OSCEs to assess gastroenterology (GI) fellows. We have successfully assessed performance across numerous milestones. Typically, OSCEs are held in person, however the COVID-19 pandemic has precipitated the need for virtual learning. We accordingly transitioned to a virtual zoom OSCE (Z-OSCE) and evaluated trainees' perception of this program.
Method(s): Fourteen first- and second-year GI fellows from five programs across multiple states participated in a four-station virtual OSCE on Zoom. Afterwards, participants answered a survey to share their perspectives and provide feedback. Learners were asked to rate the usefulness of the virtual OSCE and compare it to other in-person and virtual educational modalities. These questions were rated on a 10-point Likert scale (Figure 1). Additionally, free-text responses regarding any aspect of the OSCE were evaluated for comments on the virtual format.
Result(s): In comparing the usefulness of the virtual OSCE to other in-person modalities, trainees rated it a mean of 7.15 (range 5-10), and 31% of respondents rated it a 9 or 10. Trainees rated the virtual OSCE compared to other virtual learning modalities a mean of 8.15 (range 5-10), and 43% rated it 9 or 10. When asked whether they would recommend this OSCE as a training tool, the trainees gave a mean recommendation of 7.77 (range 5-10), and 38% gave a 9 or 10. General feedback regarding the nature of the OSCE noted the virtual format worked well, orientation to the format was important and could be improved by providing it in an email beforehand.
Conclusion(s): Virtual learning has been necessary during the COVID-19 pandemic, and it is crucial to evaluate the value of the novel Z-OSCE. Participants found the virtual OSCE may be more useful than in-person learning modalities and it compared favorably to other virtual learning modalities. One benefit of this modality was the easier inclusion of fellows from geographically disparate areas negating the need to travel for this program, a benefit given lack of universal access to simulation using standardized patients. To improve future exams, orientation prior to the day of the OSCE may improve trainees' experiences.

BACKGROUND: Healthcare systems rose to the challenges of COVID-19 by creating or expanding telehealth programs to ensure that patients could access care from home. Traditionally, though, physicians receive limited formal telemedicine training, which made preparedness for this transition uneven. We designed a survey for General Internal Medicine (GIM) physicians within our diverse health system to describe experiences with providing virtual patient care; with the ultimate goal of identifying actionable recommendations for health system leaders and medical educators.
METHOD(S): Surveys were sent to all faculty outpatient GIM physicians working at NYU Langone Health, NYC Health + Hospitals/Bellevue and Gouverneur, and the VA NY Harbor Health System (n=378) in May & June of 2020. Survey items consisted of Likert and open-ended questions on experience with televisits (13 items) and attitudes toward care (24 items). Specific questions covered barriers to communication over remote modalities.
RESULT(S): 195/378 (52%) responded to the survey. 96% of providers reported having problems establishing a connection from the patient's end while 84% reported difficultly establishing connection on the provider's end. Using interpreter services over the phone was also problematic for providers, with 38% reporting troubles. Regarding teamness, 35% of physicians found it difficult to share information with healthcare team members during virtual visits and 42% found it difficult to work collaboratively with team members, both when compared to in-person visits. When subdivided, 24% of private and 40% of public providers found info sharing more difficult (p<0.04). 31% of private providers and 45% of public found team collaboration more difficult (ns). Physicians also identified challenges in several domains including physical exams (97%), establishing relationships with new patients (74%), taking a good history (48%), and educating patients (35%). In thematic analysis of open-ended comments, themes emerged related to technological challenges, new systems issues, and new patient/provider communication experiences. Positives noted by physicians included easier communication with patients who often struggle with keeping in-person appointments, easier remote monitoring, and a more thorough understanding of patients' home lives.
CONCLUSION(S): Provider experience differences were rooted in the type of technology employed. Safety-net physicians conducted mostly telephonic visits while private outpatient physicians utilized video visits, despite both using the same brand of electronic medical record system. As we consider a new normal and prolonged community transmission of COVID-19, it is essential to establish telemedicine training, tools, and protocols that meet the needs of both patients and physicians across diverse settings. LEARNING OBJECTIVE #1: Describe challenges and barriers to effective communication and clinical skill utilization during televisits LEARNING OBJECTIVE #2: Conceptualize recommendations for educational curricula and health service improvement areas

BACKGROUND: Performance-based assessment & feedback during medical training is essential for a successful transition before moving onto residency and independent clinical practice. Learners at New York University's School of Medicine (NYUSOM) participate in a routine comprehensive clinical skills examination (CCSE) that takes place at the tail end of medical school. During this exam, learners interact with standardized patients (SPs) and are rated on specific skills using a standardized checklist, measuring important clinical skills domains. NYUSOM has utilized the same assessment tool since 2005. To date, there is limited evidence on the tool's validity and ability to differentiate among students. We sought to provide evidence for it's reliability, validity, and generalizability.
METHOD(S): 1157 learners participated in the CCSE from 2011-2019 and were included in the analysis. Communication domain items assessed included patient education (3 items), relationship development (4 items), information gathering (6 items) and organization/ time management (3 items). Items were scored using a 3-point behaviorally-anchored scale (not, partly, or well done). In order to determine the degree to which the data mapped onto our theoretically-informed communication domains, we conducted a four-factor confirmatory factor analysis (CFA) allowing for factors to correlate (oblique rotation) and using means and variance adjusted weighted-least squares estimation (WLSMV) in order to account for the ordered categorical nature of the communication items. Model fit was assessed using root mean square of approximation (RMSEA) < 0.08, comparative fit index (CFI) > 0.95, and standardized root mean square error (SRMR) <0.08.
RESULT(S): The model fit the data using RMSEA (0.04), CFI (0.98), and SRMR (0.05). All factors were significantly correlated with one another (p < 0.05), with the largest correlation between patient education and organization/ time management (0.86), and information gathering (0.77). The smallest correlation was between organization/ time management and information gathering (0.66). All items (factor loadings) significantly loaded on the factors they measured. Only one item had an insignificant threshold loading between partly and well done, suggesting this part of the response scale may be hard for SPs to differentiate between students with varying ability on this item. Each factor had at least one item that had a factor loading less than 0.7.
CONCLUSION(S): The analysis suggests each item on the communication checklist significantly measures domains they were designed to measure, and that items can be summated to compute overall scores. Domains had one item with a lower loading than the rest, suggesting these items may be measuring something different. Follow up measurement modeling and profile analysis is the next logical step in determining if there is an important sub-domain that identifies a student group operating differentially. LEARNING OBJECTIVE #1: Understand clinical communication LEARNING OBJECTIVE #2: Describe communication measures

BACKGROUND: The transition from medical school to residency is characterized by an abrupt transition of learning needs and goals. Coaching is a promising intervention to support individual learning and growth trajectories of learners. It is uncommon for medical school faculty to have undergone training as coaches. We explored our faculty's perceptions and skills after instituting a new coaching program.
METHOD(S): Faculty advisors (N=12) and GME (N=16) participated in a coaching development program and in community of practice meetings where challenging coaching scenarios were shared. GME faculty also participated in a Group Objective Structured Clinical Exam (GOSCE) to practice and receive feedback on their skills. Peer-faculty observers and resident raters used behaviorally grounded checklists to assess faculty performance. We conducted 2 focus groups: 1) UME advisors engaged in longitudinal coaching (n=9) and 2) GME faculty participating in the coaching development program (n=8) to better understand how faculty make sense of and put into practice these new coaching roles and skills.
RESULT(S): Simple thematic coding showed that both groups emphasized the blurring of the many roles they serve when interacting with trainees and struggled with recognizing both which hat to wear (role to adopt) and which skills to call upon in specific situations. UME advisors who have dedicated advising/coaching roles reported assuming multiple roles at different times with their same students. Many of the GME coaches serve as Associate Program Directors, and described adopting a coaching frame of reference (mentality) and requiring external reinforcement for coaching skills. Some reported realizing after the fact that coaching would have been a valuable approach. Faculty newer to their role felt more successful in engaging in coaching mindset and coaching. Faculty were curious about how trainees would feel about this approach and anticipated that some would appreciate this more than others. 12 faculty participated in a three station Coaching GOSCE. Both resident raters and faculty peer raters suggested faculty coaches were able to establish trust and engage in authentic listening. Coaches negotiated the tension between empathetic listening with supporting goal-setting. Residents provided slightly lower ratings than peer observers on coaches' ability to ask questions and assume a coachee- focused agenda.
CONCLUSION(S): Medical educators may benefit from obtaining coaching skills, but deliberate training in how these skills complement, and differ, from existing skills requires both didactic and experiential learning. Cultivating a community of practice and offering opportunities for deliberate practice, observation and feedback is essential for medical educators to achieve mastery as coaches. LEARNING OBJECTIVE #1: Identify and perform appropriate learning activities to guide personal and professional development (PBL) LEARNING OBJECTIVE #2: Understand and apply core longitudinal coaching skills (Professionalism)

BACKGROUND: New York City was the epicenter of the COVID pandemic in the US during early 2020. NYU Langone Medical Center was one of many New York medical centers that experienced an unprecedented influx of patients. During the onset of the pandemic, clinic leadership identified, oriented, and rapidly deployed a COVID Army, consisting of non-hospitalist physicians, to meet the needs of this patient influx. Orientation and training included an hour-long session with an emphasis on the inpatient electronic medical record system and a plan for at the elbow assistance from senior hospitalists. Here, we share feedback from our providers on our capacity building process and use information gathered to offer specific lessons learned in planning for workforce mobilization.
METHOD(S): A 32-item survey was distributed from March-June of 2020 in order to assess the experiences of these ancillary physicians, all of which were NYU Langone providers. Items included a mix of Likert and open-ended questions on demographics and attitudes toward experiences on the COVID team.
RESULT(S): All 272 volunteers received a survey. 67% (n=183) responded. 84 (46%) were from the Department of Medicine, the remainder were primarily from surgical, pediatrics or obstetrics/gynecology. Respondents worked in combination ambulatory/inpatient practices (n=94; 52%) or outpatient only (n=85; 47%) (Mean years in practice: 7.18). 76% felt that the number of patients they were in charge of felt Just Right (average: 7). 10% rated the experience as challenging (n=17). On their perception of support and training, 94% and 63% rated the support and training they received as somewhat or very effective, respectively. 89% (n=99) and 96% (n=107) of supplemental attendings felt valued and valuable to their team, respectively. 87% of respondents identified as being willing to volunteer again. In review of open-ended feedback, we identified a series of themes surrounding areas for improvement. These include the need to 1) invest time into orientations, including training on EHR use, (2) clarify roles and workflow within each team up front, (3) balance team workload if possible, (4) keep teams updated on evolving policies and recommendations, (5) make team members feel valued and supported, and (6) ensure they have the right tools available.
CONCLUSION(S): Given what we have learned from our survey, the continued waxing and waning of community infection, and the unknown length and extent of the COVID pandemic, we recommend providing transparent leadership, frequent communication, and an educational series to ensure everyone is learning together. In addition, clarity is essential, and it is important to be specific in defining the exact roles of ancillary physicians. It is our hope that the lessons learned from our needs assessment can be applied to other hospitals currently in the throes of a surge of COVID inpatients. LEARNING OBJECTIVE #1: Identify best practices for preparing an ancillary workforce for patient surge. LEARNING OBJECTIVE #2: Understand tools for quality patient care

STATEMENT OF PROBLEM OR QUESTION (ONE SENTENCE): In order to describe quality improvement (QI) methods for health systems, we report on 10-years of using Unannounced Standardized Patient (USP) visits as the core of a program of education, training, and improvement in a system serving vulnerable patients in partnership with an academic medical center. LEARNING OBJECTIVES 1: Consider methods for supporting learning healthcare systems LEARNING OBJECTIVES 2: Identify performance data to improve care DESCRIPTION OF PROGRAM/INTERVENTION, INCLUDING ORGANIZATIONAL CONTEXT (E.G. INPATIENT VS. OUTPATIENT, PRACTICE OR COMMUNITY CHARACTERISTICS): The IOM defines a Learning Healthcare System (LHCS) as one in which science, informatics, incentives and culture are aligned for continuous improvement and innovation and where best practices are seamlessly embedded in the delivery process and new knowledge is captured as an integral by-product of the delivery experience. As essential as electronic health records are to LHCS, such data fail to capture all actionable information needed to sustain learning within complex systems. USPs are trained actors who present to clinics, incognito, to portray standardized chief complaints, histories, and characteristics. We designed and delivered USP visits to two urban, safety net clinics, focusing on assessing physician, team, and clinical micro system functioning. MEASURES OF SUCCESS (DISCUSS QUALITATIVE AND/OR QUANTITATIVEMETRICSWHICHWILL BE USEDTOEVALUATE PROGRAM/INTERVENTION): Behaviorally anchored assessments are used to assess core clinical skills (e.g., communication, information gathering, patient education, adherence to guidelines, patient centeredness, and patient activation). Team functioning assessments include professionalism and coordination. Micro system assessment focuses on safety issues like identity confirmation, hand washing, and navigation. Data from these visits has been provided to the residency, primary care teams, and to leadership and have been used to drive education, team training, and QI. FINDINGS TO DATE (IT IS NOT SUFFICIENT TO STATE FINDINGS WILL BE DISCUSSED): 1111 visits have been sent to internal medicine and primary care residents and their teams/clinics. At the resident level, needs for additional education and training in depression management, opioid prescribing, smoking cessation, and patient activation were identified and informed education. Chart reviews found substantial variation in ordering of labs and tests. At the team level, USPs uncovered needs for staff training, enhanced communication, and better processes for eliciting and documenting Social Determinants of Health (SDoH). Audit/feedback reports on provider responses to embedded SDoH combined with targeted education/resources, were associated with increased rates of eliciting and effectively responding to SDoH. In the early COVID wave, USPs tested clinic response to a potentially infectious patient. Currently, USPs are being deployed to understand variability in patients' experience of telemedicine given the rapid transformation to this modality. Finally, generalizable questions about underlying principles of medical education and quality improvement are being asked & answered using USP data to foster deeper understanding of levers for change. KEY LESSONS FOR DISSEMINATION (WHAT CAN OTHERS TAKE AWAY FOR IMPLEMENTATION TO THEIR PRACTICE OR COMMUNITY): A comprehensive USP program can provide unique insights for driving QI and innovation and help sustain a LHCS

LEARNINGOBJECTIVES 1: Interpersonal /Communication Skill: 1) Identify communication skills needed to recruit older adults LEARNING OBJECTIVES 2: 2) Assess feasibility of GOSCEs to enhance recruitment skills in RAs. SETTING AND PARTICIPANTS: Convenience sample of 18 (5 male, 13 female) Research Assistants (RAs) at an urban hospital who recruit older adults for clinical trials. DESCRIPTION: Increasing older adults' participation in clinical trials is urgently needed. We developed a remote, three station simulation (Group Objective Structured Clinical Exam - GOSCE) to teach RAs communication skills. This 2-hour course included a discussion of challenges in recruiting older adults; skills practice with Standardized Participants (SPs); and a debrief to review experiences, highlight best practices. After discussion, RAs rotated (3 per group) through the stations, each with SP and faculty observer who provided immediate feedback. Thus, learners had opportunities for active and observational learning.Scenarios were: 1) an older white woman with hearing impairment; 2) an older white woman and family member together; and 3) an older Black man mistrustful due to history of racism in medical research. SPs completed behaviorally anchored checklists (11 communication skills across all cases, and 5-7 case-specific questions). Learners completed a 36- item survey of self-assessed change in skill after the workshop; insights on recruitment practice; and educational value. EVALUATION: The communication checklist across all cases included: relationship development (5 items, mean of 58% well done (range: 50-75%), patient education (3 items, 44% (42-58%)), patient satisfaction (2 items, 54% (50-58%)), and information gathering (1 item, 92%). Seventeen RAs completed the survey, 100% felt the workshop provided valuable feedback and taught relevant material, 88% would participate again and 52%reported that the workshop improved their recruitment skills. All RAs reported encountering situations similar to hearing impairment and family member cases, and the majority rated the cases as high in educational value. Just 45% reported experiencing a case similar to the Black male case, and 100% rate it as high in educational value. Key points identified by RAs included the value of building a trusting relationship with potential subjects, recognizing possible barriers to communication early on and addressing these directly in a supportive and respectful style. DISCUSSION / REFLECTION / LESSONS LEARNED: Remote GOSCEs are a feasible mechanism for training RAs in subject recruitment focused on the unique needs of older adults. Responses to the RA survey suggest that GOSCEs are feasible for training RAs in simulated clinical scenarios with which participants are familiar and unfamiliar. SP assessment of RAs identified areas for further reinforcement to improve recruitment skills. This innovation is a feasible, high yield strategy for training research staff. It is highly adaptable to the specific recruitment needs and skills of a clinical trials and will add to the literature on educating RAs

BACKGROUND: Primary care (PC) residency training is a period that provides opportunity to develop skills required for independent practice. Unannounced Standardized Patients (USPs), or secret shoppers, are a controlled measure of clinical skills in actual practice. We sought to describe differences in core clinical communication skills over the last decade for PC residents.
METHOD(S): USPs presented as a new patient for a comprehensive visit while portraying one of six unique, outpatient cases (with either chronic or acute symptomology). Actors received extensive training to ensure accurate case portrayal. Each completed a post-visit, behaviorally anchored checklist (not, partly, or well done) in order to provide extensive, actionable feedback. A standardized checklist was used, consisting of individual items across domains including information gathering, relationship development, patient education, activation and satisfaction. Chronbach's alpha for domains ranged from 0.62- 0.89. Summary scores (mean % well done) were calculated by domain and compared by year for all learners and by PGY within year for the primary care (PC) residency. Differences were assessed using ANOVA. Case portrayal accuracy was ensured using audio tape review.
RESULT(S): 396 visits were conducted with PC residents in our urban, safetynet hospital system between 2013 and 2020. While looking across the 8 years, there was variation in mean scores per domain, though Kruskal-Wallis H test did not show any statistical difference. Relationship development and info gathering were the highest rated skills, at 75% and 76% well done, respectively, on average. Patient satisfaction and activation remained uniformly low across years, with scores averaging 36% and 39% well done, respectively. Multi-variate analysis showed no significant changes across domains by cohort (grad year) and PGY levels. Further, there were no significant differences by PGY year or cohort in terms of scoring using a two-way ANOVA, though there was a slight upward trend in relationship development skills since 2017 for all PGY levels. There were similar trends in most domains, with 2020 scores being higher than previous years. There were no significant differences across domains while looking at PGY1 learners only.
CONCLUSION(S): While there were no significant differences in scores, we can postulate that PC residents enter the residency with consistent foundational communication skills, possibly attributable to training. We elected to use the visit itself as the unit of analysis, which does not allow us to tease out differences in individual learners. We also have small sample sizes for earlier years of the USP visit program, which may hinder results. Regardless, results warrant further research in order to gain a more thorough understanding, possibly in relation to curricular trends. Further study will look at individual resident differences and ideally provide insight into curricular improvement areas. LEARNING OBJECTIVE #1: Describe assessment measures LEARNING OBJECTIVE #2: Explore clinical competency

BACKGROUND: During the rapid onset of the pandemic, clinicians transitioned from traditional outpatient practice to telemedicine for triaging COVID-19 patients and providing routine care to patient panels. Telemedicine training and assessment had not been systematically incorporated into most residencies. In response, a scheduled Internal Medicine (IM) Objective Structured Clinical Examination (OSCE) was adapted to a telemedicineemphasized, virtual modality to become a just-in- time learning experience for trainees.
METHOD(S): Remote cases deployed on common web-based video conference platform included; (1) a potential COVID-19 triage, (2) educating on buprenorphine maintenance, (2) counselling on mammogram screening, and (3) addressing frustration with electronic health record documentation. Simulated Patients (SPs) rated residents on communication skills, patient activation and satisfaction, and case-specific telemedicine items. Analyses included a comparison of domain scores (mean % well done) for residents who participated in both the 2020 remote and 2019 in-person OSCEs, and a review of written resident feedback.
RESULT(S): Fewer than half (46%) of 2020's residents (n=23) performed well on the COVID-19 case's telemedicine skill domain. Residents excelled in using nonverbal communication to enrich on-camera communication (100%), but struggled with virtual physical exams (13%), gathering information (4%), and optimizing technology (4%). Residents expressed interest in more opportunity to practice telemedicine skills going forward. Residents' overall COVID-19 knowledge was fair (54% of items were rated as 'well done'). Fewer than half (45%) advised the SP that testing was not available at the time, but that he should call the city hotline for information, and about half (55%) provided quarantine/ home care instructions. In comparing 2020 (virtual) to 2019 (in-person) OSCE scores, residents who participated in both assessments (n=9) performed similarly on communication skills including information gathering (84% vs. 83%), and relationship development (93% vs. 92%), patient satisfaction (72% vs. 80%) and activation (65% vs. 66%). Patient education scores were significantly lower during the virtual OSCE (40% vs. 76%, P=.008).
CONCLUSION(S): Our reformulated OSCE accomplished three goals: (1) physically distancing residents from SPs, (2) providing residents the opportunity to practice critical telemedicine skills, and (3) alerting our educators to curricular improvement areas in virtual physical exam, patient education, gathering information and optimizing technology. Our methods are scalable at other institutions and have applications to the larger medical and clinical education community. LEARNING OBJECTIVE #1: Describe challenges and barriers to effective communication and clinical skill utilization during televisits. LEARNING OBJECTIVE #2: Understand resident physician practice patterns and communication regarding infectious disease