Faculty Bibliography

PROBLEM: Efforts to evaluate and optimize the effectiveness of medical education have been limited by the difficulty of designing medical education research. Longitudinal, epidemiological views of educational outcomes can help overcome limitations, but these approaches require "bigger data"-more learners, sources, and time points. The rich data institutions collect on students and residents can be mined, however, ethical and practical barriers to using these data must first be overcome. APPROACH: In 2008, the authors established the Research on Medical Education Outcomes (ROMEO) Registry, an educational data registry modeled after patient registries. New York University School of Medicine students, residents, and fellows provide consent for routinely collected educational, performance, quality improvement, and clinical practice data to be compiled into a deidentified, longitudinal database. As of January 2015, this registry included 1,225 residents and fellows across 12 programs (71% consent rate) and 841 medical students (86% consent rate). Procedures ensuring voluntary informed consent are essential to ethical enrollment and data use. Substantial resources are required to provide access to and manage the data. OUTCOMES: The registry supports educational scholarship. Seventy-two studies using registry data have been presented or published. These focus on evaluating the curriculum, quality of care, and measurement quality and on assessing needs, competencies, skills development, transfer of skills to practice, remediation patterns, and links between education and patient outcomes. NEXT STEPS: The authors are working to integrate assessment of relevant outcomes into the curriculum, maximize both the quantity and quality of the data, and expand the registry across institutions.

INTRODUCTION: People with a mental health diagnosis have high rates of tobacco use and encounter limited availability of tobacco treatment targeted to their needs. This study compared the effectiveness of a specialized telephone smoking-cessation intervention developed for mental health patients with standard state quit-line counseling. DESIGN: RCT. SETTING/PARTICIPANTS: The study was conducted at six Veterans Health Administration facilities in the Northeast U.S. Participants were 577 mental health clinic patients referred by their providers for smoking-cessation treatment. INTERVENTION: From 2010 to 2012, the study implemented a telephone program that included patient referral from a mental health provider, mailed cessation medications, and telephone counseling. Participants were randomized to receive a specialized multisession telephone counseling protocol (n=270) or transfer to their state's quit-line for counseling (n=307). MAIN OUTCOME MEASURES: Participants completed telephone surveys at baseline, 2 months, and 6 months. The study's primary outcome was self-reported 30-day abstinence at 6 months. Secondary outcomes were self-reported 30-day abstinence, counseling satisfaction and counseling content at 2 months, and self-reported use of cessation treatment and quit attempts at 6 months. Logistic regression was used to compare treatment groups on outcomes, controlling for baseline cigarettes per day and site. Inverse probability weighting and multiple imputation were used to handle missing abstinence outcomes. Data were analyzed in 2014-2015. RESULTS: At 6 months, participants in the specialized counseling arm were more likely to report 30-day abstinence (26% vs 18%, OR=1.62, 95% CI=1.24, 2.11). There was no significant group difference in abstinence at 2 months (18% vs 14%, OR=1.31, 95% CI=0.49, 3.49). Participants in the specialized arm were more likely to be assisted with developing a quit plan; receive follow-up calls after quitting; and receive counseling on several domains, including motivation, confidence, smoking triggers, coping with urges, and mental health symptoms (all p<0.05). Specialized counseling participants were more satisfied with treatment and more likely to find the counseling useful (p<0.05). CONCLUSIONS: The specialized counseling intervention was more effective at helping patients quit than transfer to a state quit-line. Patients were more satisfied with the specialized counseling program. TRIAL REGISTRATION: This study is registered at www.clinicaltrials.gov NCT00724308.

OBJECTIVE: The Test of Integrated Professional Skills (TIPS) is an objective structured clinical examination-simulation hybrid examination that assesses resident integration of technical, cognitive, and affective skills in Obstetrics and Gynecology (OBGYN) residents. The aim of this study was to analyze performance patterns and reactions of residents to the test to understand how it may fit within a comprehensive assessment program. DESIGN: A retrospective, mixed methods review of the design and implementation of the examination, patterns of performance of trainees at different levels of training, focus group data, and description of use of TIPS results for resident remediation and curriculum development. SETTING: OBGYN residents at New York University Langone Medical Center, a tertiary-care, urban academic health center. PARTICIPANTS: OBGYN residents in all years of training, postgraduate year-1 through postgraduate year, all residents completing the TIPS examination and consenting to participate in focus groups were included. RESULTS: In all, 24 residents completed the TIPS examination. Performance on the examination varied widely among individuals at each stage of training, and did not follow developmental trends, except for technical skills. Cronbach alpha for both standardized patient and faculty ratings ranged from 0.69 to 0.84, suggesting internal consistency. Focus group results indicated that residents respond to the TIPS examination in complex ways, ranging from anxiety about performance to mixed feelings about how to use the data for their learning. CONCLUSION: TIPS assesses a range of attributes, and can support both formative and summative evaluation. Lack of clear developmental differences and wide variation in performance by learners at the same level of training support the argument for individualized learning plans and competency-based education.

BACKGROUND:A handful of medical schools in the U.S. are awarding medical degrees after three years. While the number of three-year pathway programs is slowly increasing there is little data on the opinions of medical education leaders on the need for shortening training. PURPOSE/OBJECTIVE:To survey deans and program directors (PDs) to understand the current status of 3-year medical degree programs and to elicit perceptions of the need for shortening medical school and the benefits and liabilities of 3-year pathway programs (3YPP). METHODS:Online surveys were emailed to the academic deans of all U.S. medical schools and to a convenience sample of residency and fellowship PDs. Frequency distributions are reported for key survey items and content analysis was used to describe open-ended responses. RESULTS:Of the respondents, 7% have a 3YPP, 4% were developing one, and 35% were considering development. In 2014, 47% of educational deans and 32% of PDs agreed that there may be a need to shorten medical school. From a list of benefits, both deans and PDs agreed that the greatest benefit to a 3YPP was debt reduction (68%). PDs and deans felt reduced readiness for independence, reduced exposure to complementary curricula regarding safety and quality improvement, premature commitment to a specialty, and burnout were all potential liabilities. From a list of concerns, PDs were concerned about depth of clinical exposure, direct patient care experience, ability to assume increased responsibility, level of maturity, and certainty regarding career choice. CONCLUSIONS:Over one-third of medical schools are considering the development of a 3YPP. While there may be benefits for a select group of students, concerns regarding maturity, depth of clinical exposure, and competency must be addressed for these programs to be well received.

Program/Project Purpose: The New York University School of Medicine (NYUSOM) has been engaging medical students in global health across their years in medical school through its' International Health Program (IHP) since 2002. All students in good academic standing are supported to participate in culturally meaningful and socially relevant research in any location that meets US State Department safe travel advisories. Students can also participate in self-funded clinical rotations. The program aims to increase the cultural competency of medical graduates as they enter the workforce to serve an increasingly diverse patient population. Our assumption was that IHP participants would have increased cultural competency, increased engagement with underserved populations, and increased incorporation of global health in their career paths. Structure/Method/Design: To understand the impact of this program we conducted a simple cross-sectional cohort study of graduates of the program from 2002 to 2012. Survey questions included demographic data as well as subjective impact of the IHP program on their residency and career choices. Outcome & Evaluation: We received 49 responses out of 213 surveys distributed by email to past participants of the IHP. Analysis of the data showed that international experiences later in medical school, rather than earlier, had a higher impact on career plans (mean of 2.70 verses mean of 2.00). Moreover, electives with both research and clinical components positively impacted career plans. Clinical rotations appeared to have slightly more impact on cultural competency than research rotations (mean of 3.82 verses mean of 3.39). Students who had not studied abroad previously (n = 24) reported that IHP had a greater impact on cultural competence, commitment to global health, and commitment to caring for the under-served than those who had previously studied abroad (n = 24). Going Forward: International heath experiences for medical students appear to have a significant impact on career paths as well as improving cultural competency. These findings can potentially benefit human resources for health by increasing US graduates long-term engagement in global public health and for working with underserved populations. We feel there is also strong argument for supporting first time international experiences for medical students

ISSUE/OBJECTIVE:Medical educators and educational researchers continue to improve their processes for managing medical student and program evaluation data using sound ethical principles. This is becoming even more important as curricular innovations are occurring across undergraduate and graduate medical education. Dissemination of findings from this work is critical, and peer-reviewed journals often require an institutional review board (IRB) determination. APPROACH/METHODS:IRB data repositories, originally designed for the longitudinal study of biological specimens, can be applied to medical education research. The benefits of such an approach include obtaining expedited review for multiple related studies within a single IRB application and allowing for more flexibility when conducting complex longitudinal studies involving large datasets from multiple data sources and/or institutions. In this paper, we inform educators and educational researchers on our analysis of the use of the IRB data repository approach to manage ethical considerations as part of best practices for amassing, pooling, and sharing data for educational research, evaluation, and improvement purposes. IMPLICATIONS/CONCLUSIONS:Fostering multi-institutional studies while following sound ethical principles in the study of medical education is needed, and the IRB data repository approach has many benefits, especially for longitudinal assessment of complex multi-site data.