Faculty Bibliography

The increasing use of technology in health care and health professions education is an invitation to examine how digital sources of evidence are used in making assessment claims. In this paper, we describe how four sets of terms-primary and secondary data; structured and unstructured data; development and use; and deterministic and generative-can aid in examining how data from digital sources are used in evaluating what learners know and can do. Drawing on multiple examples, this paper shows how the four sets of terms can help both developers and users of technology-based assessment systems.

INTRODUCTION AND PROBLEM STATEMENT/UNASSIGNED:A chief resident's role incorporates administrative, academic, and interpersonal responsibilities essential to managing a successful residency program. However, rising chief residents receive little formal exposure to leadership training. OBJECTIVES/UNASSIGNED:To (1) define leadership styles; (2) understand the effect of cultural competence on leadership styles; (3) learn effective methods to advocate as the chief resident; (4) provide effective peer feedback; (5) provide effective supervisor feedback; (6) learn effective conflict management; (7) ensure psychological safety. METHODS AND CURRICULUM DESCRIPTION/UNASSIGNED:We developed a 1-day curriculum combining didactics and simulation activities for our program's rising chief residents. Implementation of our curricular design included a morning session focusing on small groups and didactic-based lectures on specific topics pertinent to leadership, along with a debriefing of a psychometric evaluation tool administered before the curriculum day. The simulation activity consisted of 3 group objective structured clinical examination (G-OSCE) scenarios: (1) providing a struggling junior trainee with feedback; (2) debriefing an adverse clinical outcome as the team leader; (3) navigating a challenging situation with a supervising physician. Standardized participants were surveyed for specific objectives. Learners completed precurricular and postcurricular surveys on their familiarity and preparedness for their chief year. RESULTS AND ASSESSMENT DATA/UNASSIGNED:= 0.421), learner-reported use of wellness resources was noted to be reduced after the curricular intervention and remains a result of further interest for exploration. DISCUSSION AND LESSONS LEARNED/UNASSIGNED:A 1-day leadership development curriculum combining didactics and simulation is an effective means of preparing rising chief residents to succeed in their transition to this leadership role.

BACKGROUND AND OBJECTIVE/UNASSIGNED:Incidence rates for prostate cancer (PCa) diagnosis and mortality are higher for Black men. It is unknown whether similar disparities exist in survivorship care. We assessed the delivery and quality of survivorship care for Black men undergoing PCa therapy in terms of the burden of and treatment for urinary adverse events (UAEs) and erectile dysfunction (ED). METHODS/UNASSIGNED:We queried Optum Clinformatics data for all patients diagnosed with PCa from January 1, 2002 to December 31, 2017 and identified those who underwent primary PCa treatment. Index cohorts were identified in each year and followed longitudinally until 2017. Data for UAE diagnoses, UAE treatments, and ED treatments were analyzed in index cohorts. Cox proportional-hazards regression models were used to examine associations of race with UAE diagnosis, UAE treatment, and ED treatment. KEY FINDINGS AND LIMITATIONS/UNASSIGNED:We identified 146, 216 patients with a PCa diagnosis during the study period, of whom 55, 149 underwent primary PCa treatment. In the primary treatment group, 32.7% developed a UAE and 28.2% underwent UAE treatment. The most common UAEs were urinary incontinence (11%), ureteral obstruction/stricture (4.5%), bladder neck contracture (4.5%), and urethral stricture (3.7%). The most common UAE treatments were cystoscopy (13%), suprapubic tube placement (6%), and urethral dilation (5%). Overall, UAE diagnosis rates were higher for Black patients, who had significantly higher risk of urethral obstruction, rectourethral fistula, urinary incontinence, cystitis, urinary obstruction, and ureteral fistula. Overall, UAE treatment rates were lower for Black patients, who had significantly higher risk of fecal diversion and/or rectourethral fistula repair (adjusted hazard ratio [aHR] 1.71, 95% confidence interval [CI] 1.04-2.79). Regarding ED treatments, Black patients had higher risk of penile prosthesis placement (aHR 1.591, 95% CI 1.26-2.00) and intracavernosal injection (aHR 1.215, 95% CI 1.08-1.37). CONCLUSIONS AND CLINICAL IMPLICATIONS/UNASSIGNED:Despite a high UAE burden, treatment rates were low in a cohort with health insurance. Black patients had a higher UAE burden and lower UAE treatment rates. Multilevel interventions are needed to address this stark disparity. ED treatment rates were higher for Black patients. PATIENT SUMMARY/UNASSIGNED:We reviewed data for patients treated for prostate cancer (PCa) and found that 32.7% were diagnosed with a urinary adverse event (UAE) following their PCa treatment. The overall treatment rate for these UAEs was 28.2%. Analysis by race showed that the UAE diagnosis rate was higher for Black patients, who were also more likely to receive treatment for erectile dysfunction.

In competency-based medical education (CBME), which is being embraced globally, the patient-learner-educator encounter occurs in a highly complex context which contributes to a wide range of assessment outcomes. Current and historical barriers to considering context in assessment include the existing post-positivist epistemological stance that values objectivity and validity evidence over the variability introduced by context. This is most evident in standardized testing. While always critical to medical education the impact of context on assessment is becoming more pronounced as many aspects of training diversify. This diversity includes an expanding interest beyond individual trainee competence to include the interdependency and collective nature of clinical competence and the growing awareness that medical education needs to be co-produced among a wider group of stakeholders. In this Eye Opener, we wish to consider: 1) How might we best account for the influence of context in the clinical competence assessment of individuals in medical education? and by doing so, 2) How could we usher in the next era of assessment that improves our ability to meet the dynamic needs of society and all its stakeholders? The purpose of this Eye Opener is thus two-fold. First, we conceptualize - from a variety of viewpoints, how we might address context in assessment of competence at the level of the individual learner. Second, we present recommendations that address how to approach implementation of a more contextualized competence assessment.

PURPOSE/OBJECTIVE:As the COVID-19 pandemic forced most colleges and universities to go online, student health centers rapidly shifted to telehealth platforms without frameworks for virtual care provision. An urban student health center implemented a needs assessment involving unannounced standardized patients (USPs) to evaluate the integration of a new telehealth workflow and clinicians' virtual communication skills. METHODS:From April to May 2021, USPs conducted two video visits with 12 primary care and four women's health clinicians (N = 16 clinicians; 32 visits). Cases included (1) a 21-year-old female presenting for birth control with a positive Patient Health Questionaire-9 and (2) a 21-year-old male, who vapes regularly, with questions regarding safe sex with men. Clinicians were evaluated using a checklist completed by the USP immediately following the visit and a systematic chart review of the electronic health record. RESULTS:USP feedback indicates most clinicians received high ratings for general communication skills but may benefit from educational intervention in several key telemedicine skills. Clinicians struggled with using nonverbal signals to enrich communication (47% well done), acknowledging emotions (34% well done), and using video for information gathering (34% well done). Low rates of standard screenings (e.g., 63% administered the PHQ-2, <50% asked about alcohol use) suggested protocols for in-person care were not easily incorporated into telehealth practices, and clinicians may benefit from enhanced care team support. Performance reports were shared with clinicians and leadership postvisit. DISCUSSION/CONCLUSIONS:Results suggest project design and implementation is scalable and feasible for use at other institutions, offering a structured methodology that can improve general student health care.

Discrimination toward trainees is a pervasive problem, with surveys showing it is often perpetrated by patients. For several years, residents and faculty in an internal medicine residency have participated in a workshop offering a framework for responding to discriminatory behavior by patients. As part of a larger effort to reinforce this teaching and promote an inclusive environment, the authors pursued a multi-pronged simulation curriculum that could be incorporated into graduate medical education programs across their institution. First, the authors conducted trainee and faculty focus groups to better understand their experiences. Qualitative data was collected, including recommendations for responding to discrimination, characteristics of the most common experiences, and trainees' own ideas for worthwhile simulation scenarios. Trainees and faculty were then brought together in a collaborative process to co-develop simulation cases that were later implemented in curricula across multiple learner levels, specialties, and contexts. Participants in these simulations reported improved comfort in responding to discrimination in the moment and/or in the wake of such incidents. Through trainee-faculty collaboration, the project yielded authentic and impactful simulation experiences for learners, while also giving trainees an opportunity to turn previous trauma into constructive learning opportunities that promote an inclusive environment.

OBJECTIVES/OBJECTIVE:Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS:) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS:Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS/CONCLUSIONS:This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.

OBJECTIVE:This pilot study compared a novel communication strategy, the positive approach to the psychiatric interview, with the traditional approach to see if the positive approach can be taught to psychiatric residents; reproduced with standardized patients; measured with a structured scale, the "Positive Approach Outcome Measure," by blinded raters; and used to improve rapport (assessed with the Bond score), a key driver of engagement. METHODS:Thirty psychiatric residents were randomly assigned to conduct two psychiatric interviews with standardized patients. The standardized patients completed the Working Alliance Inventory-Short Revised, an assessment of the therapeutic alliance. T tests and linear regression examined the effect of the training on the outcome of interest, the Bond score. RESULTS:The Bond scores for the positive approach group (M = 19.27, SD = 2.87) and the traditional approach group (M = 16.90, SD = 3.44) were statistically significantly different (p = 0.05). All residents trained in the positive approach received a positive score on the Positive Approach Outcome Measure while none of the traditional approach-trained residents attained the threshold. The inter-rater reliability for the blinded raters was high (0.857), as was the intra-rater reliability (1.0). CONCLUSIONS:The positive approach can be taught to residents and reproduced consistently and was associated with improvement in a key driver of treatment engagement: rapport. The positive approach may be an important, inexpensive intervention to improve treatment engagement and ultimately treatment outcomes.