Faculty Bibliography

The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.

Learning curves can support a competency-based approach to assessment for learning. When interpreting repeated assessment data displayed as learning curves, a key assessment question is: "How well is each learner learning?" We outline the validity argument and investigation relevant to this question, for a computer-based repeated assessment of competence in electrocardiogram (ECG) interpretation. We developed an on-line ECG learning program based on 292 anonymized ECGs collected from an electronic patient database. After diagnosing each ECG, participants received feedback including the computer interpretation, cardiologist's annotation, and correct diagnosis. In 2015, participants from a single institution, across a range of ECG skill levels, diagnosed at least 60 ECGs. We planned, collected and evaluated validity evidence under each inference of Kane's validity framework. For Scoring, three cardiologists' kappa for agreement on correct diagnosis was 0.92. There was a range of ECG difficulty across and within each diagnostic category. For Generalization, appropriate sampling was reflected in the inclusion of a typical clinical base rate of 39% normal ECGs. Applying generalizability theory presented unique challenges. Under the Extrapolation inference, group learning curves demonstrated expert-novice differences, performance increased with practice and the incremental phase of the learning curve reflected ongoing, effortful learning. A minority of learners had atypical learning curves. We did not collect Implications evidence. Our results support a preliminary validity argument for a learning curve assessment approach for repeated ECG interpretation with deliberate and mixed practice. This approach holds promise for providing educators and researchers, in collaboration with their learners, with deeper insights into how well each learner is learning.

Simulation is a widely used technique for medical education. Due to decreased training opportunities with real patients, and increased emphasis on both patient outcomes and remote access, demand has increased for more advanced, realistic simulation methods. Here, we discuss the increasing need for, and benefits of, extended (virtual, augmented, or mixed) reality throughout the continuum of medical education, from anatomy for medical students to procedures for residents. We discuss how to drive the adoption of mixed reality tools into medical school's anatomy, and procedural, curricula.

Medical educators are not yet taking full advantage of the publicly available clinical practice data published by federal, state, and local governments, which can be attributed to individual physicians and evaluated in the context of where they attended medical school and residency training. Understanding how graduates fare in actual practice, both in terms of the quality of the care they provide and the clinical challenges they face, can aid educators in taking an evidence-based approach to medical education. Although in their infancy, efforts to link clinical outcomes data to educational process data hold the potential to accelerate medical education research and innovation. This approach will enable unprecedented insight into the long-term impact of each stage of medical education on graduates' future practice. More work is needed to determine best practices, but the barrier to using these public data is low and the potential for early results is immediate. Using practice data to evaluate medical education programs can transform how the future physician workforce is trained and better align continuously learning medical education and health care systems.

Background: ECG interpretation is an important competency for medical students. Existing methods of teaching and learning tend to be unsystematic and poorly assessed. The dominant educational strategy is usually time-based (all students complete a common program with variable outcomes) instead of competency-based (students complete a variable program up to a common competency standard). In an online learning environment, the educator can individualize practice. We present the results of a pilot trial where we compare time- and competency-based approaches. Methods: We introduced the competency-based innovation midway through 6 consecutive EM rotations. Students practiced ECG interpretation using a case bank of 293 cases. A single diagnosis was assigned to each case by double cardiologist over-reads. 50% of the cases were normal; the abnormal cases were made up of: BBB, Pericarditis, Ventricular Hypertrophy, Ischemia and Infarction. Students interpreted each case and were given immediate feedback. In the first 3 rotations, each learner was assigned to complete 75 cases; after reviewing those results, we determined a "competency threshold" and assigned the subsequent 3 rotations of students to practice until they were able to achieve a 75% correct standard. We compared the two cohorts in terms of overall accuracy, terminal competency (last 16 cases done) and number of ECGs completed. We also present the overall item difficulty statistics. Results: 67 students participated in the study, 27 in the first 3 rotations. On average each student completed 110 ECGs, well above the assigned minimum (range 21-293). There were no statistically significant differences by rotation group (1-3 versus 4-6) in terms of : * overall accuracy (67% v. 65%; 95%CI diff: -0.02, +0.06) * number of ECGs done (108 v. 111; 95%CI diff: -29,+21) * slope of the learning curve (p=NS). There was variability in which ECG diagnoses were difficult for the students in terms of overall percent correct: Normal 84% correct; BBB 76%; STEMI 60%; Ischemia 48%; Pericarditis 42%; Ventricular Hypertrophy 40%. Conclusions: In this pilot study of ECG skill in medical students, changing to a competency threshold did not change immediate outcomes. Any effect of the change may have been washed away by the fact that the students practiced ECG interpretation at a frequency above the minimum requirements

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

The Institute of Medicine identified interprofessional education (IPE) as a key innovation for achieving the triple aim of better care, better outcomes and reduced health care costs. Yet, a shortage of qualified faculty and difficulty with aligning learners' schedules often prevent sustainable and scalable IPE. A virtual IPE intervention was developed to circumvent these barriers and compared to a blended-learning IPE intervention. We used a pre-test and post-test design with two comparison interventions to test the effects of these IPE interventions on changes in teamwork knowledge, skills and attitudes. The interventions were delivered to pre-licensure learners at a large, metropolitan medical and a nursing school. We used one-sample and independent-sample t-tests to analyze data from 220 learners who received the blended-learning intervention in 2011 and 540 learners who received the virtual learning intervention in 2012. The students in the blended-learning intervention did not significantly (p < 0.05) outperform the students in the virtual learning intervention for any of the measured outcomes, except for medical students' attitudes around team value. Virtual IPE learning is an effective, scalable, and sustainable solution for imparting foundational teamwork knowledge in health profession students.