Faculty Bibliography

Introduction/UNASSIGNED:Despite a decline, smoking rates have remained high, especially in communities with lower income, education, and limited insurance options. Evidence shows that physician-initiated counseling on smoking cessation is effective and saves lives, and that specific skills are needed to appropriately lead this type of patient-physician communication. Residency is a critical moment for future physicians and may be the optimal time to learn, practice, and refine this skillset. Unannounced Standardized Patients (USPs) have been found to be effective, incognito evaluators of resident practices. Methods/UNASSIGNED:This study introduced rigorously trained actors (USPs) into two urban, safety-net clinics to assess resident ability to engage, activate, and counsel a pre-contemplative smoker. A complementary chart review assessed appropriate documentation in the patient's electronic health record (EHR) and its relationship to counseling style and prescribing practices. Results/UNASSIGNED:Resident scores (% well done) on patient education and engagement were low (33% and 23%, respectively). Residents who coupled cessation advice with an open discussion style activated their patients more than those who solely advised cessation across all comparable measures. On EHR documentation, residents who accurately documented smoking history were more likely to directly advise their patient to quit smoking when compared to residents who did not document (t(97) = 2.828, p = .006, Cohen's D = 0.56). Conclusions/UNASSIGNED:Results highlight the need to reinforce training in patient-centered approaches including motivational interviewing, counseling, and shared decision-making. Future research should focus on the effects of smokers in pre-contemplation on physician counseling style and examine the relationship between medical training and provider communication to guide interventions.

Background: Moral distress occurs when the ethically correct action cannot be taken because of internal or external constraints. High levels of moral distress are associated with burnout in medical providers. A better understanding of factors associated with high moral distress during post-graduate training is crucial to executing interventions to lower rates of burnout during residency.
Method(s): This is a mixed Methods prospective observational cohort study that aims to enroll 90 internal medicine (IM) residents on a rolling basis from one IM residency program (12/2018-12/2019). Data is being collected by a series of 3 surveys over 1 year, using the previously validated Moral Distress Scale, to evaluate frequency and intensity of distress associated with specific situations experienced or witnessed by residents during training. This initial analysis, bivariate and multivariate regression of quantitative data from the first time point (survey #1), investigated associations between moral distress scores and demographic, training-specific, and intrinsic personal factors of participants. Analysis of qualitative open-ended questions further explored causes of moral distress and as well as coping mechanisms employed by residents.
Result(s): 32 IM residents (44% PGY-1, 28% PGY-2 and 28% PGY-3) have been enrolled thus far. 53% of resident participants identify as male, 43% as female (and 3% as other). 66% were " Categorical" residents, 25% " Primary Care (PC)," and 9.4% " Preliminary Year" PGY-1 Interns. Mean and median moral distress scores were: 66.8 (SD 31.0) and 61 (range 16-132). In multivariate linear regression " PC" residents had scores 31 pt. higher compared to " Categorical" residents (p=0.009). Male residents had scores 25pt lower than female residents (p=0.008), and PGY year conferred an incremental score increase of 11 pt. per year (p=0.057). The model was adjusted for covariates: PGY-year, gender, age, and/or program type. Themes regarding causes of moral distress included: lack of resources, situations when patient care is dictated by cost-saving measures, and aggressive futile care. Coping mechanisms included: debriefing with team members or others outside of work, active individual reflection, exercise/yoga/meditation, participating in activities and social events outside of medicine, reflective writing/journaling, and suppression and/or distraction.
Conclusion(s): In this preliminary analysis, residents in the PC track have higher average moral distress scores. It is unclear whether residents prone to more moral distress self select into this track or whether distress is related to differences in training between PC and categorical tracks. Additionally, more senior residents had average higher scores. This supports the theory of residual moral distress; an increasing amount of moral distress is experienced as a provider witnesses/experiences distressing events over time. Most coping strategies involve social connection and reflection

Background: For quality care, physicians must be skilled in diagnosing and treating chronic pain. Some studies have shown gender differences in how providers manage pain. And more broadly, female providers provide more patient-centered communication which in turn has been linked to patient activation and satisfaction with care. We explore, using Unannounced Standardized Patients (USPs), whether resident physician gender is associated with the core underlying skills needed to effectively diagnose and management chronic pain: communication, patient centeredness, and patient activation.
Method(s): We designed two USP cases and sent these undercover patients into primary care clinics at two urban, safety-net clinics. The USP cases were similar: a 30-35 y.o. male, presented as a new patient to the clinic with either shoulder pain induced by heavy lifting or knee pain due to a recreational sports injury. USPs completed a post-visit checklist that assessed patient satisfaction (4 items), patient activation (3 items), and communication skills (13 items) using a behaviorally-anchored scale (not done or partly done vs. well done). Summary scores were calculated for each of the three domains. Residents provided consent for their educational data to be used for research as part of an IRB-approved medical education registry.
Result(s): A total of 135 USP visits (80 female providers, 55 male) occurred between 2012 and 2018. Female providers saw 41 shoulder pain and 39 knee pain cases while male providers saw 21 shoulder and 34 knee cases. ANOVA was used to assess differences in summary scores by provider gender (male vs female) and by case portrayed (knee vs shoulder). Skills did not differ significantly by whether knee or shoulder pain case. Gender effects were not seen for patient centeredness or for patient activation; however female providers performed significantly better at relationship development (83% vs males 72% shoulder pain; 70% vs 66% knee pain case; p<.001) and information gathering (86% vs. males 72% shoulder pain; 79% vs66% in knee case; p<.016). Male providers, however, performed slightly better in patient education and counseling (65% vs 63% for shoulder and 38% vs 33% for knee cases; p<.001).
Conclusion(s): Developing a relationship and gathering information are critical to pain management and female residents performed better than male residents in these areas. Male providers performed slightly better than women in patient education and provider gender was not associated with any differences in patient centeredness or activation. In the future, we plan to link these underlying skills to pain management decisions, documentation and ultimately to patient outcomes. We suspect that patient activation may best be measured at follow-up, something not possible with our current USP methodology. Gender differences could be viewed as striking in the context of our relatively homogeneous sample (medicine residency program) and shared clinical environment/healthcare system

Background: Asthma education should focus on patient self-management support. Objective structured clinical examinations (OSCEs), as measured by standardized patients (SPs), provide a controlled, simulated setting for timed competency assessments while Unannounced Standardized Patients (USPs) measure clinical skills transfer in real world clinical settings. Both enable us to assess skills critical for providing quality care to patients. Learners seeing USPs have added real world stressors such as clinical load. This study describes differences in education and activation skills in two assessment contexts.
Method(s): A cohort of primary care residents (n=20) were assessed during two time points: an OSCE and a USP visit at an urban, safety-net clinic from 2009-2010. Residents consented to use of their de-identified routine educational data for research. The SP and USPs presented with the same case; a female asthmatic patient with limited understanding of illness management and concern over symptom exacerbation. Providers were rated using a behaviorally-anchored checklist upon visit completion. Competency domains assessed included patient education (4 items) and activation (4 items). Within the education domain, items included illness management, while the activation domain items assessed resident communication/counseling style. Responses were scored as not done or well done. Summary scores (mean % well done) were calculated by domain. OSCE vs USP means were compared using a paired samples t-test.
Result(s): Residents were more likely to offer an oral steroid as treatment in the OSCE case (50% vs. 35% for USPs), but performed better with USPs on most other items including domain scores. Residents seeing a USP scored significantly higher on five out of eight individual assessment items (p<.05) including recommending a spacer, helping a patient understand their condition, making patients feel like they can take control of their own health, helping a patient understand illness management, and having a patient leave feeling confident in finding solutions independently. Inhaler technique was assessed rarely in either setting (OSCE: 15%, USP: 5%). Domain summary scores (% well done) from the OSCE (activation: 12%, education: 31%) were lower than USP scores (activation: 84%, education: 37%), with differences in overall activation scores being significant (t(19)=-8.905, p<.001).
Conclusion(s): OSCEs are a widely accepted tool for measuring resident competency in a standardized environment but may be focused primarily on knowledge and technical skills. While SPs are trained to be as objective as possible, rater bias might impact scores. USPs may provide more nuanced assessments of communication skills in a setting with reduced time constraints. Next steps include examining attitudes toward OSCEs vs the clinical setting, looking at impact of provider gender, and examining setting-specific issues that promote or hinder high quality care

Background: Inter-provider variability is a major source of low-value care. The dissemination of clinical practice guidelines (CPGs) has targeted this variability, yet 44% of physicians are non-adherent to CPG. This may be due to factors including exceptionalism and incentive misalignment that present a conflict between comprehensiveness and prudence in work-up. A subset of super-utilizers are notable outliers: fewer than 0.5% of physicians account for 10% of healthcare costs. Super-utilizers order labs, request consults, order imaging, and prescribe medications at rates 30%, 140%, 14%, and 25% higher than the general population. We sought to quantify provider-specific low-value test ordering behaviors across three cases.
Method(s): Unannounced standardized patients (USPs) were trained for standardized simulation of three clinical scenarios: a "Well" visit, a chief complaint of "Fatigue," and a diagnosis of "Asthma." USPs were introduced into medicine residents' clinics in a large urban, safety-net hospital. Diagnostic orders were extracted via retrospective chart review. Scenario-specific appropriateness of diagnostic testing was determined by referencing United States Preventative Services Task Force (USPSTF) and specialty society CPGs. "Excessive" tests were those not explicitly indicated for a given scenario in either USPSTF or society CPGs (versus "indicated" tests). "Discretionary" tests were those conditionally indicated, pending patient-specific factors (such as hemoglobin A1C, pending BMI).
Result(s): One or more excessive tests were ordered in 44%, 22%, and 17% of Well (n=124), Fatigue (n=148), and Asthma (n=148) encounters respectively. Percent of orders that were excessive were 18%, 8%, and 10%, respectively. On average, 1.3 (+/-1.7) excessive orders were made. Within each case, rates of excessive ordering were positively correlated with rates of indicated and discretionary ordering, and negatively correlated with rates of omitting indicated tests. For example, in Fatigue, the correlation between excessive and indicated orders was 0.38, between excessive and discretionary orders rates was 0.59, and between excessive and omitted-indicated tests was-0.25 (all p< 0.05). A similar, statistically-significant pattern was found for the other two cases. 10 (21%) and 4 (8%) of 48 residents completing all scenarios demonstrated excessive ordering at rates atleast 1 and 2 standard deviations above the mean, respectively.
Conclusion(s): Introducing USPs representing clinical scenarios revealed marked inter-provider variability. Positive associations between rates of excessive, discretionary and indicated ordering suggest tendencies for comprehensiveness over prudence. Over one-fifth of residents completing all 3 cases were high-utilizers, and nearly one in ten were super-utilizers. Awareness of provider-level ordering tendencies can guide education and interventions supporting appropriate diagnostic use

Background: Diagnostic tests account directly for 5% of healthcare costs, but influence decisions constituting 70% of health spending. Only 5% of ordered labs are actually " high value," depending on clinical circumstances. Low-value tests, defined as not appropriate for a given clinical scenario, are ordered in one in five clinic visits. Up to $ 750 billion is spent on these low-value tests, contributing to the estimated one-quarter to one-third of healthcare spending is on wasteful services. We sought to quantify test-specific low-value ordering behaviors in urban outpatient clinics across three standardized patient cases.
Method(s): Unannounced standardized patients (USPs-highly trained actors portraying patients with standardized case presentations) were introduced into medicine residents' primary care clinics in a large urban, safety net hospital over the past five years. The USPs simulated three common outpatient clinical scenarios: a " Well" visit, a visit with a chief complaint of " Fatigue," and a visit with a diagnosis of " Asthma." Diagnostic orders were extracted via retrospective chart review for these standardized visits. For each scenario, appropriateness of diagnostic testing was determined by reference to United States Preventative Services Task Force (USPSTF) and relevant specialty society clinical practice guidelines (CPGs). " Wasteful" (over-ordered) tests were defined as those not explicitly indicated for the given scenario. Costs were derived from GoodRx.com according to local ZIP codes.
Result(s): The most commonly wasteful tests for the Asthma case were CBC (8% of 170 visits) and Chem-7 (6%), though the relative risk of over-ordering TSH was 3.8x that of other scenarios. The most commonly over-ordered tests for the Fatigue case were LFTs (14% of 148 visits) and HBV (5%), with LFTs ordered up to 15-fold more frequently than in other scenarios. The most commonly over-ordered tests for the Well case were BMP (35% of 124 visits), CBC (15%), LFTs (15%), and HBV (11%) ordered at rates up to 6.3x, 2.0x, 14.2x, and 7.4x higher than other scenarios. Finally, the average per patient excess costs were $ 8.27 (+/-$ 1.76), $ 6.79 (+/-$ 4.5), and $ 23.5 (+/-$ 9.34) for Asthma, Fatigue, and Well cases respectively.
Conclusion(s): Inappropriateness in test ordering patterns were observed through USP simulated cases. Certain tests (CBC, BMP, LFTs, and HBV) were more likely used wastefully across cases. Between cases, specific tests were ordered in an inappropriate manner (such as TSH for Asthma, LFTs for Fatigue, and BMP for Well visits). The per patient direct cost of low value testing rose above $ 20 per visit for the Well visit, though the Fatigue case exhibited the most variation. Notably, this excludes downstream (indirect) costs inestimatable from standardized encounters alone. Knowledge of wasteful utilization patterns associated with specific clinical scenarios can guide interventions targeting appropriate use of testing

Background: The core Entrustable Professional Activities medical school graduates should be able to perform on day 1 of residency provides a framework for readiness for residency. Communication skills are an essential foundation for these core EPAs and yet there have been few studies that describe communication competence across the UME-GME continuum. We report on our OSCE-based assessment of communication skills from the first few weeks of medical school to the first year of medicine residency.
Method(s): Assessment of communication is consistent in our OSCE program across UME and GME. Domains include Information Gathering (5 items), Relationship Development (6 items), and Patient Education (3 items) and these are assessed via a behaviorally anchored checklist (scores=% well done) that has strong reliability and validity evidence. In this study, we report on 3 multi-station OSCEs: the Introductory Clinical Experience (ICE) OSCE that occurs within the first weeks of medical school; the high-stakes, pass/fail Comprehensive Clinical Skills Examination (CCSE) OSCE that is fielded after clerkship year; and the Medicine Residency Program's PGY 1 OSCE. Across 3 classes of medical school (2014-2016) we have complete data for the 24 students who continued on in our Medicine Residency (and who provided consent to include their educational data in an IRB-approved registry). Analyses focus on differences in communication skills over time and between cohorts and the relationship between communication skills measured in medical school and those assessed in residency.
Result(s): Communication scores show significant improvement through medical school (but not into residency) in Information gathering (ICE mean=56%; CCSE mean=76%; PGY1 mean=77%) (F=11.54, p<.001, ICE< CCSE) and in relationship development (ICE=59%; CCSE=78%; PGY1= 74% (F=10.68, p<.001, ICE < CCSE). Mean patient education skills, however, increase significantly across all 3 time points (32% to 50% to 65%; F=31.00, p<.001). Patterns are similar across cohorts except that the Class of 2016 means increase from CCSE to PGY1. Regression analyses show that CCSE information gathering scores are more strongly associated with PGY performance than ICE scores (Std Beta=.32 vs.06), while for relationship development, it is the ICE scores that are more strongly associated (Std Beta=.40 vs.24). ICE and CCSE patient education skills have associations with PGY1 skill of similar size (Std Beta=.30 and.28).
Conclusion(s): Findings, despite the small sample, suggest a clear developmental trajectory for communication skills development and that information gathering and patient education skills may be more influenced by medical school than relationship development. That communication skills seem to level out in PGY1 highlights need for re-consolidation as clinical complexity increases. Results can inform theory development on how communication skills develop and point to transitions where skills practice/feedback may be particularly important

BACKGROUND:Few programs train residents in recognizing and responding to distressed colleagues at risk for suicide. AIM/OBJECTIVE:To assess interns' ability to identify a struggling colleague, describe resources, and recognize that physicians can and should help colleagues in trouble. SETTING/METHODS:Residency programs at an academic medical center. PARTICIPANTS/METHODS:One hundred forty-five interns. PROGRAM DESIGN/UNASSIGNED:An OSCE case was designed to give interns practice and feedback on their skills in recognizing a colleague in distress and recommending the appropriate course of action. Embedded in a patient "sign-out" case, standardized health professionals (SHP) portrayed a resident with depressed mood and an underlying drinking problem. The SHP assessed intern skills in assessing symptoms and directing the resident to seek help. PROGRAM EVALUATION/RESULTS:Interns appreciated the opportunity to practice addressing this situation. Debriefing the case led to productive conversations between faculty and residents on available resources. Interns' skills require further development: while 60% of interns asked about their colleague's emotional state, only one-third screened for depression and just under half explored suicidal ideation. Only 32% directed the colleague to specific resources for his depression (higher among those that checked his emotional state, 54%, or screened for depression, 80%). DISCUSSION/CONCLUSIONS:This OSCE case identified varying intern skill levels for identifying and assessing a struggling colleague while also providing experiential learning and supporting a culture of addressing peer wellness.

Background: Considerable resources are put into training physicians to be effective providers after residency. Practicing physicians are generally assumed to be more effective and more efficient than resident physicians who are still undergoing training. We capitalize on a unique opportunity to test that hypothesis using the controlled methodology of Unannounced Standardized Patients (USPs), Standardized Patients sent into clinical environments to systematically assess provider skills in the context of a standardized clinical scenario. Due to last minute scheduling changes, a small sample of attending physicians ended up seeing USPs we had intended to send to residents. In this study, we report on comparisons between how these attending physicians performed in terms of their patient centeredness, patient activation, assessment, and communication skills in comparison to residents.
Method(s): 6 USP visits were delivered to primary care clinics in an urban safety net hospital from 2009 to 2015. Of those 700+ visits, visits were completed inadvertently with 16 attendings. We selected the 16 attendings with at least 4 years of post-graduate experience and then matched them with 2 resident visits based on hospital, time period, and USP visit type (n=32 residents). In all visits, USPs completed a behav-iorally anchored post-visit checklist that assessed patient centeredness (4 items), patient activation (2 items), visit-specific assessment (10 items), and communication skills including information gathering (4 items), relationship development (5 items) and patient education (3 items). Items were rated as not done or partially done vs. well done and summary scores were calculated as % well done. Mean scores for attendings and matched residents were compared using t-tests.
Result(s): Resident and attending scores on patient centeredness (68% vs 73%), patient activation (44% vs 38%), assessment (53% vs 51%), patient education (49% vs 52%), information gathering (71% vs 78%) and relationship development (70% vs 73%) did not significantly differ (p>.05). Nor did we see any substantial differences in variances or find any outliers.
Conclusion(s): In our matched sample of residents and attendings, there were no significant differences by training level for any of the assessed clinical skills. While we viewed the inadvertent scheduling of USP visits with attendings as an opportunity to investigate the impact of training, our study is limited by the small sample size and whether we were able to create good matches. Findings may reflect ceiling effects (our checklists are too hard) or expertise-reversal effects (experts can skip some elements of the interaction and still arrive at the correct diagnosis and treatment plan). Further research, if our mistakenly-assessed attending sample increases, could explore the influence of PGY level and of patient load as attendings carry substantially heavier patient panels and see more (and probably more complex) patients per day then residents