Try a new search

Format these results:

Searched for:

person:ak41

in-biosketch:yes

Total Results:

245


Validity evidence for the clinical communication skills assessment tool (CCSAT) from 9 years of implementation in a high stakes medical student OSCE

Ark, Tavinder; Kalet, Adina; Tewksbury, Linda; Altshuler, Lisa; Crowe, Ruth; Wilhite, Jeffrey; Hardowar, Khemraj; Zabar, Sondra; Gillespie, Colleen
OBJECTIVES/OBJECTIVE:Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS:) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS:Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS/CONCLUSIONS:This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.
PMID: 38851013
ISSN: 1873-5134
CID: 5668672

Measuring the development of a medical professional identity through medical school

Lusk, P; Ark, T; Crowe, R; Monson, V; Altshuler, L; Harnik, V; Buckvar-Keltz, L; Poag, M; Belluomini, P; Kalet, A
PURPOSE/UNASSIGNED:The Professional Identity Essay (PIE) is a theory and evidence-based Medical Professional Identity Formation (MPIF) measure. We describe trajectories of PIE-measured MPIF over a 4-year US medical school curriculum. METHODS/UNASSIGNED:Students write PIEs at medical school orientation, clinical clerkships orientation, and post-advanced (near graduation) clerkship. A trained evaluator assigns an overall stage score to narrative responses to nine PIE prompts (inter-rater ICC 0.83, 95% CI [0.57 - 0.96], intra-rater ICC 0.85). Distribution of PIE stage scores across time points were analyzed in the aggregate and individual students were classified as Increase, Stable (no score change) or Decrease based on the trajectories of PIE stage scores over time. RESULTS/UNASSIGNED: CONCLUSIONS/UNASSIGNED:Medical students' PIE stage scores increase over time with three distinctive trajectories. Further study is needed to explore the utility of this method for formative assessment, program evaluation, and MPIF research.
PMID: 37917985
ISSN: 1466-187x
CID: 5655422

Racial Implicit Bias and Communication Among Physicians in a Simulated Environment

Gonzalez, Cristina M; Ark, Tavinder K; Fisher, Marla R; Marantz, Paul R; Burgess, Diana J; Milan, Felise; Samuel, Malika T; Lypson, Monica L; Rodriguez, Carlos J; Kalet, Adina L
IMPORTANCE/UNASSIGNED:Racial implicit bias can contribute to health disparities through its negative influence on physician communication with Black patients. Interventions for physicians to address racial implicit bias in their clinical encounters are limited by a lack of high-fidelity (realistic) simulations to provide opportunities for skill development and practice. OBJECTIVE/UNASSIGNED:To describe the development and initial evaluation of a high-fidelity simulation of conditions under which physicians might be influenced by implicit racial bias. DESIGN, SETTING, AND PARTICIPANTS/UNASSIGNED:This cross-sectional study, performed on an online platform from March 1 to September 30, 2022, recruited a convenience sample of physician volunteers to pilot an educational simulation. EXPOSURES/UNASSIGNED:In the simulation exercise, physicians saw a 52-year-old male standardized patient (SP) (presenting as Black or White) seeking urgent care for epigastric pain, nausea, and vomiting. The case included cognitive stressors common to clinical environments, including clinical ambiguity, stress, time constraints, and interruptions. Physicians explained their diagnosis and treatment plan to the SP, wrote an assessment and management plan, completed surveys, and took the Race Implicit Association Test (IAT) and Race Medical Cooperativeness IAT. The SPs, blinded to the purpose of the study, assessed each physician's communication using skills checklists and global rating scales. MAIN OUTCOMES AND MEASURES/UNASSIGNED:Association between physicians' IAT scores and SP race with SP ratings of communication skills. RESULTS/UNASSIGNED:In 60 physicians (23 [38.3%] Asian, 4 [6.7%] Black, 23 [38.3%] White, and 10 [16.7%] other, including Latina/o/x, Middle Eastern, and multiracial; 31 [51.7%] female, 27 [45.0%] male, and 2 [3.3%] other), the interaction of physicians' Race IAT score and SP race was significant for overall communication (mean [SD] β = -1.29 [0.41]), all subdomains of communication (mean [SD] β = -1.17 [0.52] to -1.43 [0.59]), and overall global ratings (mean [SD] β = -1.09 [0.39]). Black SPs rated physicians lower on communication skills for a given pro-White Race IAT score than White SPs; White SP ratings increased as physicians' pro-White bias increased. CONCLUSIONS AND RELEVANCE/UNASSIGNED:In this cross-sectional study, a high-fidelity simulation calibrated with cognitive stressors common to clinical environments elicited the expected influence of racial implicit bias on physicians' communication skills. The outlined process and preliminary results can inform the development and evaluation of interventions that seek to address racial implicit bias in clinical encounters and improve physician communication with Black patients.
PMID: 38506811
ISSN: 2574-3805
CID: 5640562

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine

Smirnova, Alina; Chahine, Saad; Milani, Christina; Schuh, Abigail; Sebok-Syer, Stefanie S; Swartz, Jordan; Wilhite, Jeffrey A; Kalet, Adina; Durning, Steven J; Lombarts, Kiki M J M H; van der Vleuten, Cees P M; Schumacher, Daniel J
PURPOSE/OBJECTIVE:Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). METHOD/METHODS:EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified. RESULTS:274 residents and 1,963 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations (r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. CONCLUSIONS:For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.
PMID: 36351056
ISSN: 1938-808x
CID: 5357362

Understanding medical student paths to communication skills expertise using latent profile analysis

Altshuler, Lisa; Wilhite, Jeffrey A.; Hardowar, Khemraj; Crowe, Ruth; Hanley, Kathleen; Kalet, Adina; Zabar, Sondra; Gillespie, Colleen; Ark, Tavinder
Purpose: To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students. Materials and methods: Performance data from 1182 consenting third year medical students in 9 cohorts (2011"“2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items. Results: Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner"™s skill profiles. Conclusion: Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.
SCOPUS:85150984589
ISSN: 0142-159x
CID: 5460042

Measuring the development of a medical professional identity through medical school

Lusk, P.; Ark, T.; Crowe, R.; Monson, V.; Altshuler, L.; Harnik, V.; Buckvar-Keltz, L.; Poag, M.; Belluomini, P.; Kalet, A.
Purpose: The Professional Identity Essay (PIE) is a theory and evidence-based Medical Professional Identity Formation (MPIF) measure. We describe trajectories of PIE-measured MPIF over a 4-year US medical school curriculum. Methods: Students write PIEs at medical school orientation, clinical clerkships orientation, and post-advanced (near graduation) clerkship. A trained evaluator assigns an overall stage score to narrative responses to nine PIE prompts (inter-rater ICC 0.83, 95% CI [0.57 − 0.96], intra-rater ICC 0.85). Distribution of PIE stage scores across time points were analyzed in the aggregate and individual students were classified as Increase, Stable (no score change) or Decrease based on the trajectories of PIE stage scores over time. Results 202 students completed 592 PIEs from 2018-2023. There was a significant change in the proportion of PIEs in stages over time (X2 84.40, p < 0.001), 47% (n = 95) students were categorized in the Increase trajectory, 45.5% (n = 92) as Stable and 7.4% (n = 15) as Decrease. Older age and time-predicted stage scores change within trajectories (p < 0.05). Conclusions Medical students"™ PIE stage scores increase over time with three distinctive trajectories. Further study is needed to explore the utility of this method for formative assessment, program evaluation, and MPIF research.
SCOPUS:85175717486
ISSN: 0142-159x
CID: 5616362

Heartache or Bellyache? Epigastric Pain, Communication Skills, and Implicit Bias: Can We Uncover an Association in the Simulation Lab?

Ark, Tavinder; Fisher, Marla; Milan, Felise; Kalet, Adina L; Marantz, Paul R; Burgess, Diana; Rodriguez, Carlos J; Burd-Orama, Lily; Samuel, Malika; Gonzalez, Cristina M
PMCID:9614548
PMID: 36419761
ISSN: 1938-808x
CID: 5364222

Viewing Readiness-for-Residency through Binoculars: Mapping Competency-Based Assessments to the AAMC's 13 Core Entrustable Professional Activities (EPAs)

Eliasz, Kinga L; Nick, Michael W; Zabar, Sondra; Buckvar-Keltz, Lynn; Ng, Grace M; Riles, Thomas S; Kalet, Adina L
PMID: 35668557
ISSN: 1532-8015
CID: 5283072

Mentoring Underrepresented Minority Physician-Scientists to Success

Kalet, Adina; Libby, Anne M; Jagsi, Reshma; Brady, Kathleen; Chavis-Keeling, Deborah; Pillinger, Michael H; Daumit, Gail L; Drake, Amelia F; Drake, Wonder Puryear; Fraser, Victoria; Ford, Daniel; Hochman, Judith S; Jones, Rochelle D; Mangurian, Christina; Meagher, Emma A; McGuinness, Georgeann; Regensteiner, Judith G; Rubin, Deborah C; Yaffe, Kristine; Ravenell, Joseph E
As the nation seeks to recruit and retain physician-scientists, gaps remain in understanding and addressing mitigatable challenges to the success of faculty from underrepresented minority (URM) backgrounds. The Doris Duke Charitable Foundation Fund to Retain Clinical Scientists program, implemented in 2015 at 10 academic medical centers in the United States, seeks to retain physician-scientists at risk of leaving science because of periods of extraordinary family caregiving needs, hardships that URM faculty-especially those who identify as female-are more likely to experience. At the annual Fund to Retain Clinical Scientists program directors conference in 2018, program directors-21% of whom identify as URM individuals and 13% as male-addressed issues that affect URM physician-scientists in particular. Key issues that threaten the retention of URM physician-scientists were identified through focused literature reviews; institutional environmental scans; and structured small- and large-group discussions with program directors, staff, and participants. These issues include bias and discrimination, personal wealth differential, the minority tax (i.e., service burdens placed on URM faculty who represent URM perspectives on committees and at conferences), lack of mentorship training, intersectionality and isolation, concerns about confirming stereotypes, and institutional-level factors. The authors present recommendations for how to create an environment in which URM physician-scientists can expect equitable opportunities to thrive, as institutions demonstrate proactive allyship and remove structural barriers to success. Recommendations include providing universal training to reduce interpersonal bias and discrimination, addressing the consequences of the personal wealth gap through financial counseling and benefits, measuring the service faculty members provide to the institution as advocates for URM faculty issues and compensating them appropriately, supporting URM faculty who wish to engage in national leadership programs, and sustaining institutional policies that address structural and interpersonal barriers to inclusive excellence.
PMID: 34495889
ISSN: 1938-808x
CID: 5200092

A Preliminary Evaluation of Students' Learning and Performance Outcomes in an Accelerated 3-Year MD Pathway Program

Cangiarella, Joan; Eliasz, Kinga; Kalet, Adina; Cohen, Elisabeth; Abramson, Steven; Gillespie, Colleen
Background/UNASSIGNED:Little outcome data exist on 3-year MD (3YMD) programs to guide residency program directors (PDs) in deciding whether to select these graduates for their programs. Objective/UNASSIGNED:To compare performance outcomes of 3YMD and 4-year MD (4YMD) students at New York University Grossman School of Medicine. Methods/UNASSIGNED:In 2020, using the Kirkpatrick 4-level evaluation model, outcomes from 3 graduating cohorts of 3YMD students (2016-2018) were compared with the 4YMD counterparts. Results/UNASSIGNED:=.03), other metrics and overall intern ratings did not differ by pathway. Conclusions/UNASSIGNED:Exploratory findings from a single institution suggest that 3YMD students performed similarly to 4YMD students in medical school and the first year of residency.
PMCID:8848877
PMID: 35222827
ISSN: 1949-8357
CID: 5174042