Faculty Bibliography

Background: Low-value tests, defined as inappropriate for a given clinical scenario, are ordered in one in five clinic visits. Residents tend to over-order diagnostic tests to "minimize uncertainty" of presenting cases, even though these tests are not useful according to Bayesian statistics; a pursuit deemed the "Ulysses syndrome". Simultaneously, evidence suggests residents misuse preventive tests in half of relevant clinical scenarios. We sought to quantify ordering behaviors in urban primary care clinics across three unannounced standardized cases.
Method(s): Unannounced standardized patients (USPs) were trained for standardized simulation of three clinical scenarios: a "Well" visit, a chief complaint of "Fatigue," and a diagnosis of "Asthma." USPs were introduced into medicine residents' clinics in an urban, safety-net hospital. All electronic orders were extracted via chart review. Scenario-specific appropriateness of diagnostic testing was determined by referencing United States Preventive Services Task Force (USPSTF) and society clinical practice guidelines (CPGs). "Preventive" tests (such as lipid panels or hemoglobin A1C) were derived from USPSTF guidelines whereas "Diagnostic" tests (such as pulmonary function testing for Asthma or heterophile antibodies for Fatigue) were from CPGs. "Excessive" tests were those not explicitly indicated for a given scenario in either USPSTF or CPGs (versus "indicated" tests).
Result(s): Indicated tests were ordered in 29% of Well (124 encounters), 16% of Fatigue (148 encounters), and 12% of Asthma (170 encounters) cases. One or more excessive tests were ordered in 44%, 22%, and 17% of Well, Fatigue, and Asthma encounters respectively. The distribution of indicated and excessive tests for preventive versus diagnostic purposes varied by case: In Well visits, the majority (71%) of excessive testing was in pursuit of a diagnosis, while three-quarters of indicated testing was for preventive purposes. In Fatigue and Asthma visits, the reverse patterns were true: the majority of indicated tests ordered were diagnostic (81% and 68%, respectively) while the majority of excessive tests were preventive (78% and 63%, respectively).
Conclusion(s): Introducing USPs to resident clinics revealed that, for patients presenting without a chief complaint (Well visit), residents successfully ordered less than one-third of indicated tests, and over 75% of inappropriately ordered tests pursued a diagnosis. For patients presenting with chief complaints (Fatigue and Asthma), rates of appropriate ordering were even lower (16% and 12%), and tended to overlook preventive care. In these cases, inappropriate tests tended to be ordered for preventive purposes. Awareness of resident mis-utilization of preventive and diagnostic testing in distinct clinical circumstances can guide educational efforts towards evidence-based care and resource stewardship

Background: It is well documented that few ordered tests are " high value" a significant percentage of those ordered are " low-value." Residency offers an opportunity to teach high-value care and educational interventions to do so have been effective. However, the relationship between high-value care and residents' ability to communicate effectively with patients has not been explored. Ability to establish rapport, gather information effectively, and be patient-centered may impact residents' use of tests. We hypothesize that residents with poor skills in these areas may order tests less efficiently.
Method(s): Unannounced Standardized Patients (USPs) were introduced into residents' primary care clinics in a large urban, safety net hospital to portray 3 clinical scenarios: a well visit, a chief complaint of fatigue, and a diagnosis of asthma. Orders were extracted via chart review. Appropriateness of orders was determined by reference to United States Preventive Services Task Force (USPSTF) and clinical practice guidelines. Excessive tests were defined as not explicitly indicated for the scenario-indicated tests were the converse. Number of excessive and % of indicated tests were calculated across the 3 visits for 48 residents. Communication skills in information gathering (5 items) and developing a relationship (6 items) and a patient-centeredness score (4 items: took a personal interest, answered all my questions) were computed as % of behaviorally anchored items rated as " well done" and included in regression models predicting test utilization.
Result(s): On average, residents ordered 15% of indicated tests (SD 9%, 0-38%) across the 3 visits and a mean of 1.3 unnecessary tests (SD 1.7, 0-6). In the regression model, the 3 skills explained 16% of variation in unnecessary tests (p=.047). Information gathering explained the greatest share (8%, p=.041). With all 3 variables in the model, patient-centeredness was positively associated with unnecessary tests (Std Be-ta=.42, p=.016) and information gathering was negatively associated with unnecessary tests (Std Beta=-.34, p=.041). Mirroring these Results, superutilizers (10 residents ordering > =3 excessive tests) had lower information gathering and relationship development scores than other residents (66% vs 75% and 72% vs 76%) but higher patient centeredness scores (80% vs 74%)-although differences were not significant.
Conclusion(s): Our findings suggest that information gathering skills may have a small influence on residents' ordering of excessive tests. Further research with larger samples (adequate power) will help clarify the effect sizes. If our Results stand, interventions for high-value care should include information gathering skills and residency programs should continue to reinforce core communication skills training. In addition, our finding that patient centeredness was associated with ordering unnecessary tests suggests that residency programs could caution residents about conflating ordering of tests with patient-centeredness

Background: Asthma education should focus on patient self-management support. Objective structured clinical examinations (OSCEs), as measured by standardized patients (SPs), provide a controlled, simulated setting for timed competency assessments while Unannounced Standardized Patients (USPs) measure clinical skills transfer in real world clinical settings. Both enable us to assess skills critical for providing quality care to patients. Learners seeing USPs have added real world stressors such as clinical load. This study describes differences in education and activation skills in two assessment contexts.
Method(s): A cohort of primary care residents (n=20) were assessed during two time points: an OSCE and a USP visit at an urban, safety-net clinic from 2009-2010. Residents consented to use of their de-identified routine educational data for research. The SP and USPs presented with the same case; a female asthmatic patient with limited understanding of illness management and concern over symptom exacerbation. Providers were rated using a behaviorally-anchored checklist upon visit completion. Competency domains assessed included patient education (4 items) and activation (4 items). Within the education domain, items included illness management, while the activation domain items assessed resident communication/counseling style. Responses were scored as not done or well done. Summary scores (mean % well done) were calculated by domain. OSCE vs USP means were compared using a paired samples t-test.
Result(s): Residents were more likely to offer an oral steroid as treatment in the OSCE case (50% vs. 35% for USPs), but performed better with USPs on most other items including domain scores. Residents seeing a USP scored significantly higher on five out of eight individual assessment items (p<.05) including recommending a spacer, helping a patient understand their condition, making patients feel like they can take control of their own health, helping a patient understand illness management, and having a patient leave feeling confident in finding solutions independently. Inhaler technique was assessed rarely in either setting (OSCE: 15%, USP: 5%). Domain summary scores (% well done) from the OSCE (activation: 12%, education: 31%) were lower than USP scores (activation: 84%, education: 37%), with differences in overall activation scores being significant (t(19)=-8.905, p<.001).
Conclusion(s): OSCEs are a widely accepted tool for measuring resident competency in a standardized environment but may be focused primarily on knowledge and technical skills. While SPs are trained to be as objective as possible, rater bias might impact scores. USPs may provide more nuanced assessments of communication skills in a setting with reduced time constraints. Next steps include examining attitudes toward OSCEs vs the clinical setting, looking at impact of provider gender, and examining setting-specific issues that promote or hinder high quality care

Learning Objective #1: Recognize and manage opioid overdose in a community health center setting Learning Objective #2: Improve linkage to effective treatment for opioid use disorder after overdose CASE: A 54 yo man was found unresponsive at the door of our community health center. Rapid Response was called and the patient was found to be unresponsive to sternal rub, with 6 breaths per minute, and pinpoint pupils. One dose of 4mg of intranasal naloxone was administered, and soon the patient was alert and oriented. He declined transfer to the ED but was amenable to observation, stating that he had just purchased his usual 3 bags and used them outside of his primary care clinic. His PMH was notable for 36 years of IV/intranasal heroin use, prior stroke, GERD, glaucoma, hyperlipidemia, lumbar radiculopathy, and tobacco use. He had one overdose in the 1990s, attempted detox several times and tried self-treating himself with methadone and buprenorphine. He lived with his girlfriend and was unemployed. His medications included cyclobenzaprine and ranitidine. The medical assistant from our addiction medicine clinic engaged the patient, who reported that the overdose scared him, and offered medication treatment which he accepted. The addiction clinic nurse and physician saw the patient that day and gave an initial buprenorphine prescription, instructions and follow-up appointment. He is now stable on buprenorphine 8mg daily. IMPACT/DISCUSSION: The overdose described in this case represents one of three overdoses in the past month at our NYC health center. Urban health centers often serve as community hubs and may be seen as a safer place to use opioids. Overdoses at community health centers represent an important point of patient engagement in treatment for OUD. Treatment with opioid agonist therapy after overdose has been shown to reduce all-cause and opioid-related mortality. However, only a small percentage of patients receive medication therapy after overdose. (Larochelle et al. Annals of Internal Medicine. 2018) Initiation of medication treatment for OUD at the time of ED presentation has also been shown to improve engagement in treatment. (DOnofrio et al. JAMA 2015.) Institutional commitment to training all providers and staff to recognize the signs of opioid overdose and administer intranasal naloxone has direct impact on patient outcomes. Our experience with this case has demonstrated the importance of immediate engagement in care at the time of overdose. Having a team available at the time of overdose that cares for patients with addiction enabled us to quickly engage this patient in care and start medication therapy when he felt most receptive to treatment.
Conclusion(s): Our case demonstrates two crucial steps for improving outcomes in opioid overdose: widespread availability of and training for intranasal naloxone use, along with community health sites equipped to treat patients with opioid use disorder at the time of overdose

Background: Inter-provider variability is a major source of low-value care. The dissemination of clinical practice guidelines (CPGs) has targeted this variability, yet 44% of physicians are non-adherent to CPG. This may be due to factors including exceptionalism and incentive misalignment that present a conflict between comprehensiveness and prudence in work-up. A subset of super-utilizers are notable outliers: fewer than 0.5% of physicians account for 10% of healthcare costs. Super-utilizers order labs, request consults, order imaging, and prescribe medications at rates 30%, 140%, 14%, and 25% higher than the general population. We sought to quantify provider-specific low-value test ordering behaviors across three cases.
Method(s): Unannounced standardized patients (USPs) were trained for standardized simulation of three clinical scenarios: a "Well" visit, a chief complaint of "Fatigue," and a diagnosis of "Asthma." USPs were introduced into medicine residents' clinics in a large urban, safety-net hospital. Diagnostic orders were extracted via retrospective chart review. Scenario-specific appropriateness of diagnostic testing was determined by referencing United States Preventative Services Task Force (USPSTF) and specialty society CPGs. "Excessive" tests were those not explicitly indicated for a given scenario in either USPSTF or society CPGs (versus "indicated" tests). "Discretionary" tests were those conditionally indicated, pending patient-specific factors (such as hemoglobin A1C, pending BMI).
Result(s): One or more excessive tests were ordered in 44%, 22%, and 17% of Well (n=124), Fatigue (n=148), and Asthma (n=148) encounters respectively. Percent of orders that were excessive were 18%, 8%, and 10%, respectively. On average, 1.3 (+/-1.7) excessive orders were made. Within each case, rates of excessive ordering were positively correlated with rates of indicated and discretionary ordering, and negatively correlated with rates of omitting indicated tests. For example, in Fatigue, the correlation between excessive and indicated orders was 0.38, between excessive and discretionary orders rates was 0.59, and between excessive and omitted-indicated tests was-0.25 (all p< 0.05). A similar, statistically-significant pattern was found for the other two cases. 10 (21%) and 4 (8%) of 48 residents completing all scenarios demonstrated excessive ordering at rates atleast 1 and 2 standard deviations above the mean, respectively.
Conclusion(s): Introducing USPs representing clinical scenarios revealed marked inter-provider variability. Positive associations between rates of excessive, discretionary and indicated ordering suggest tendencies for comprehensiveness over prudence. Over one-fifth of residents completing all 3 cases were high-utilizers, and nearly one in ten were super-utilizers. Awareness of provider-level ordering tendencies can guide education and interventions supporting appropriate diagnostic use

Background: Newly recruited clinicians have heterogeneous Backgrounds and experiences and need a substantive introduction to their new institution's patient communication expectations and safety culture and standards for clinician performance. We describe a unique onboarding program designed to ensure that newly hired clinicians receive actionable, behaviorally specific feedback from the patients' perspective to support a satisfying transition to the new work environment, enhance patient experience and reduce the need to punitively react to complaints once they have started.
Method(s): During the 2-hour onboarding, participants complete 3, 10-minute Objective Structured Clinical Exam cases designed to assess how they address a medical error, manage the patient's discharge goals of care, and respond to an impaired learner. During each encounter, participants interact with highly trained Standardized Patients (SPs) or Standardized Learners (SLs) who use behaviorally-anchored checklists to evaluate provider performance on communication and case-specific skills. Following each encounter, participants complete a self-assessment while the SPs/SLs complete a behavior-specific checklist, after which the two discuss the encounter and the SL/SP provides confidential and actionable feedback. At the end, participants are encouraged to set individual learning goals to implement in their daily work, complete a program evaluation, and engage in a debrief with experienced facilitators. Participants also receive their SP checklists in addition to an institutional guide containing relevant resources and contacts.
Result(s): Over 2 years, 57 faculty members representing 6 clinical sites participated in the onboarding program. They are heterogeneous with respect to general and case specific performance on these SP/SL cases. For example, 86% adequately elicited the SP/SLs story during the discharge case compared to 66% in the other two cases, 77% addressed pain management (a key patient goal), while 44% did not discuss important medication side effects. Participants have universally found this onboarding to be useful and relevant; 98% agreed/strongly agreed that the program was an effective way to reinforce good habits in patient and learner communication, 96% felt it enhanced confidence about their ability to communicate effectively, and 96% felt it reinforced the institutional culture of safety. All 56 participants who completed the evaluation agreed/strongly agreed that the event was engaging and well-designed, and 93% felt it was a good use of their time and would recommend the program.
Conclusion(s): Traditional orientations are not well recalled and do not address knowledge and skills in real-time. Although it requires additional resources, participants are enthusiastic about our low-stakes introduction to the institution's expectations. This program sets high standards and introduces a new model for skills-based onboarding which may lead to measurably improved patient outcomes

Background: The core Entrustable Professional Activities medical school graduates should be able to perform on day 1 of residency provides a framework for readiness for residency. Communication skills are an essential foundation for these core EPAs and yet there have been few studies that describe communication competence across the UME-GME continuum. We report on our OSCE-based assessment of communication skills from the first few weeks of medical school to the first year of medicine residency.
Method(s): Assessment of communication is consistent in our OSCE program across UME and GME. Domains include Information Gathering (5 items), Relationship Development (6 items), and Patient Education (3 items) and these are assessed via a behaviorally anchored checklist (scores=% well done) that has strong reliability and validity evidence. In this study, we report on 3 multi-station OSCEs: the Introductory Clinical Experience (ICE) OSCE that occurs within the first weeks of medical school; the high-stakes, pass/fail Comprehensive Clinical Skills Examination (CCSE) OSCE that is fielded after clerkship year; and the Medicine Residency Program's PGY 1 OSCE. Across 3 classes of medical school (2014-2016) we have complete data for the 24 students who continued on in our Medicine Residency (and who provided consent to include their educational data in an IRB-approved registry). Analyses focus on differences in communication skills over time and between cohorts and the relationship between communication skills measured in medical school and those assessed in residency.
Result(s): Communication scores show significant improvement through medical school (but not into residency) in Information gathering (ICE mean=56%; CCSE mean=76%; PGY1 mean=77%) (F=11.54, p<.001, ICE< CCSE) and in relationship development (ICE=59%; CCSE=78%; PGY1= 74% (F=10.68, p<.001, ICE < CCSE). Mean patient education skills, however, increase significantly across all 3 time points (32% to 50% to 65%; F=31.00, p<.001). Patterns are similar across cohorts except that the Class of 2016 means increase from CCSE to PGY1. Regression analyses show that CCSE information gathering scores are more strongly associated with PGY performance than ICE scores (Std Beta=.32 vs.06), while for relationship development, it is the ICE scores that are more strongly associated (Std Beta=.40 vs.24). ICE and CCSE patient education skills have associations with PGY1 skill of similar size (Std Beta=.30 and.28).
Conclusion(s): Findings, despite the small sample, suggest a clear developmental trajectory for communication skills development and that information gathering and patient education skills may be more influenced by medical school than relationship development. That communication skills seem to level out in PGY1 highlights need for re-consolidation as clinical complexity increases. Results can inform theory development on how communication skills develop and point to transitions where skills practice/feedback may be particularly important

Background: Faculty development programs (FDP) in medical education can increase clinician educators' (CE) confidence in teaching and improve their teaching skills. The impact of FDP on faculty's role as educators and sense of an educator community is less well understood. Identification with a community of educators (COE) can enhance teaching in the workplace along with personal and professional growth. We evaluated the impact of participation in the Education for Educators program (E4E) on these issues. E4E is a yearlong FDP designed to enhance teaching confidence and skill in a variety of venues; improve ability to assess learners; promote an environment of academic inquiry with trainees at different levels; and create a COE.
Method(s): An annual needs assessment of key stakeholders including medical school deans, program directors, and participants forms the basis for the E4E curriculum. The program begins with a Group Observed Structured Teaching Experience (GOSTE) followed by three 3-hour workshops which pair a clinical and teaching topic. After each workshop, participants complete " commitment to change" statements and take part in peer-to-peer (P2P) observations wherein participants observe each other teaching in their usual teaching environment. The program concludes with structured debriefs and an assessment of participants' perception of their role as educators and their sense of an educator community. Participants reported how participation in E4E impacted their teaching and what new skills they implemented. Structured phone conversations assessed the same information one-year after completing the program.
Result(s): Fifty-one CEs completed the program in two cohorts (2016-17 and 2017-18), 60% of whom were women. Participants included 20 hospitalists and 31 subspecialists, averaging 8 years in practice (range 1-28) and spending an average of 63% of their time in patient care (range 10-100%). Thirty-eight participants (75%) completed the immediate post-program debrief sessions. Participants reported a renewed identification with their role as an educator. They cited a change in perspective to become more reflective and focused on teaching and recognized that their teaching skills can in fact be improved. Many reported time constraints as a barrier to teaching. They noted an increased identification with their COE, stating that they now had peers and mentors with whom to discuss teaching challenges. To date, phone interviews have been completed with three participants at one-year of follow-up. The preliminary Results show a sustained impact on educators' roles and belonging to a COE. They also reported ongoing use of specific skills including resilience strategies, and planning teaching sessions.
Conclusion(s): Longitudinal FDP in medical education for CE can lead to a greater appreciation for the role of an educator, and identification with a COE. Investment in longitudinal FDP may have lasting impact on the clinical learning environment and the identity of faculty as an educator

Background: The transition from medical student to intern presents a major patient safety concern. Our institution implemented an immersive First Night OnCall (FNOC) simulation to support transitioning trainees and cultivate a culture of safety.
Objective(s): Engage pediatric interns in a pediatric focused FNOC simulation to ensure readiness to recognize and address common safety issues in practice.
Method(s): Interns were asked to recognize patient safety hazards in a simulated patient room and participate in case based safety discussions. Interns then participated in GOSCEs (Group Observed Standardized Clinical Encounters). GOSCEs tasked trainees to obtain informed consent, evaluate a decompensating patient, recognize a mislabeled culture bottle, and give an effective patient handoff. Faculty debriefed all activities. Learners completed pre and post program assessments and a program evaluation.
Result(s): Twenty incoming interns completed FNOC. Only 11% reported any prior formal training in patient safety. Interns recognized 46% of the environmental patient safety hazards. Out of the 5 GOSCE groups, 3 called a rapid response team, 3 noted the label error for the culture bottle, and 3 obtained complete informed consent. After FNOC, 92% of interns reported increased comfort (4 or 5 on 1-5 scale) in speaking to a supervisor, escalating a situation, and reporting a medical error. All interns agreed that the case based safety discussions and the patient safety room increased readiness for internship. Almost all of the interns (85%) agreed or strongly agreed that FNOC was an effective way to learn patient safety, a good approach to improve readiness, fun, and engaging.
Conclusion(s): Incoming interns are not consistently able to demonstrate common safety practices. Engaging, immersive, simulation based experiences like FNOC may reduce this variability, while simultaneously instilling aspirational institutional norms, promoting a culture of safety, and providing a framework for effective on-boarding strategies for new trainees.
Copyright