Faculty Bibliography

Models for diagnostic reasoning in radiology have been based on the observed behaviors of experienced radiologists but have not directly focused on the thought processes of novices as they improve their accuracy of image interpretation. By collecting think-aloud verbal reports, the current study was designed to investigate differences in specific thought processes between medical students (novices) as they learn and radiologists (experts), so that we can better design future instructional environments. Seven medical students and four physicians with radiology training were asked to interpret and diagnose pediatric elbow radiographs where fracture is suspected. After reporting their diagnosis of a case, they were given immediate feedback. Participants were asked to verbalize their thoughts while completing the diagnosis and while they reflected on the provided feedback. The protocol analysis of their verbalizations showed that participants used some combination of four processes to interpret the case: gestalt interpretation, purposeful search, rule application, and reasoning from a prior case. All types of processes except reasoning from a prior case were applied significantly more frequently by experts. Further, gestalt interpretation was used with higher frequency in abnormal cases while purposeful search was used more often for normal cases. Our assessment of processes could help guide the design of instructional environments with well-curated image banks and analytics to facilitate the novice's journey to expertise in image interpretation.

STATEMENT/UNASSIGNED:The International Network for Simulation-based Pediatric Innovation, Research, and Education co-hosted a novel research accelerator meeting with the International Pediatric Simulation Society in May of 2019 in Toronto. The purpose of the meeting was to bring together healthcare simulation scientists with resuscitation stakeholders to brainstorm strategies for accelerating progress in the science of saving pediatric lives from cardiac arrest. This was achieved by working in teams to draft targeted requests for proposals calling the research community to action investigating this topic. During the 1-day meeting, groups were divided into 6 teams lead by experts representing specific domains of simulation research. Teams developed a pitch and presented a sample request for proposals to a panel of expert judges, making a case for why their domain was the most important to create a funding opportunity. The winner of the competition had their specific request for proposal turned into an actual funding opportunity, supported by philanthropy that was subsequently disseminated through International Network for Simulation-based Pediatric Innovation, Research, and Education as a competitive award. An inspired donor supported an award for the second-place proposal as well, evidence of early research acceleration catalyzed from this conference. This article is a summary of the meeting rationale, format, and a description of the requests for proposals that emerged from the meeting. Our goal is to inspire other stakeholders to use this document that leverages simulation and resuscitation science expertise, as the framework to create their own funding opportunities, further accelerating pediatric resuscitation research, ultimately saving the lives of more children worldwide.

Background/UNASSIGNED:With the implementation of competency-based assessment systems, education programs are collecting increasing amounts of data about medical learners. However, learning analytics are rarely employed to use this data to improve medical education. Objective/UNASSIGNED:We identified outstanding issues that are limiting the effective adoption of learning analytics in medical education. Methods/UNASSIGNED:Participants at an international summit on learning analytics in medical education generated key questions that need to be addressed to move the field forward. Small groups formulated questions related to data stewardship, learner perspectives, and program perspectives. Three investigators conducted an inductive qualitative content analysis on the participant questions, coding the data by consensus and organizing it into themes. One investigator used the themes to formulate representative questions that were refined by the other investigators. Results/UNASSIGNED:Sixty-seven participants from 6 countries submitted 195 questions. From them, we identified 3 major themes: implementation challenges (related to changing current practices to collect data and utilize learning analytics); data (related to data collection, security, governance, access, and analysis); and outcomes (related to the use of learning analytics for assessing learners and faculty as well as evaluating programs and systems). We present the representative questions and their implications. Conclusions/UNASSIGNED:Our analysis highlights themes regarding implementation, data management, and outcomes related to the use of learning analytics in medical education. These results can be used as a framework to guide stakeholder education, research, and policy development that delineates the benefits and challenges of using learning analytics in medical education.

Theory: Learning in digital environments allows the collection of inexpensive, fine-grained process data across a large population of learners. Intentional design of the data collection can enable iterative testing of an instructional design. In this study, we propose that across a population of learners the information from multiple choice question responses can help to identify which design features are associated with positive learner engagement. Hypothesis: We hypothesized that, within an online module that presents serial knowledge content, measures of click-level behavior will show sufficient, but variable, association with a test-measure so as to potentially guide instructional design. Method: The Aquifer online learning platform employs interactive approaches to enable effective learning of health professions content. A multidisciplinary focus group of experts identified potential learning analytic measures within an Aquifer learning module, including: hyperlinks clicked (yes/no), magnify buttons clicked (yes/no), expert advice links clicked (yes/no), and time spent on each page (seconds). Learning analytics approaches revealed which click-level data was correlated with the subsequent relevant Case MCQ. We report regression coefficients where the dependent variable is student accuracy on the Case MCQ as a general indicator of successful engagement. Results: Clicking hyperlinks, magnifying images, clicking "expert" links, and spending >100 seconds on each page were learning analytic measures and were positively correlated with Case MCQ success; rushing through pages (<20 seconds) was inversely correlated with success. Conversely, for some measures, we failed to find expected associations. Conclusions: In online learning environments, the wealth of process data available offers insights for instructional designers to iteratively hone the effectiveness of learning. Learning analytic measures of engagement can provide feedback as to which interaction elements are effective.

Objectives/UNASSIGNED:To complement bedside learning of point-of-care ultrasound (POCUS), we developed an online learning assessment platform for the visual interpretation component of this skill. This study examined the amount and rate of skill acquisition in POCUS image interpretation in a cohort of pediatric emergency medicine (PEM) physician learners. Methods/UNASSIGNED:This was a multicenter prospective cohort study. PEM physicians learned POCUS using a computer-based image repository and learning assessment system that allowed participants to deliberately practice image interpretation of 400 images from four pediatric POCUS applications (soft tissue, lung, cardiac, and focused assessment sonography for trauma [FAST]). Participants completed at least one application (100 cases) over a 4-week period. Results/UNASSIGNED:We enrolled 172 PEM physicians (114 attendings, 65 fellows). The increase in accuracy from the initial to final 25 cases was 11.6%, 9.8%, 7.4%, and 8.6% for soft tissue, lung, cardiac, and FAST, respectively. For all applications, the average learners (50th percentile) required 0 to 45, 25 to 97, 66 to 175, and 141 to 290 cases to reach 80, 85, 90, and 95% accuracy, respectively. The least efficient (95th percentile) learners required 60 to 288, 109 to 456, 160 to 666, and 243 to 1040 cases to reach these same accuracy benchmarks. Generally, the soft tissue application required participants to complete the least number of cases to reach a given proficiency level, while the cardiac application required the most. Conclusions/UNASSIGNED:Deliberate practice of pediatric POCUS image cases using an online learning and assessment platform may lead to skill improvement in POCUS image interpretation. Importantly, there was a highly variable rate of achievement across learners and applications. These data inform our understanding of POCUS image interpretation skill development and could complement bedside learning and performance assessments.

BACKGROUND:Residents are undertrained to perform acute pediatric sexual abuse evaluations. The American Academy of Pediatrics has proposed development of an adaptable child abuse curriculum, though no such curriculum exists. OBJECTIVES/OBJECTIVE:Our goal was to perform a needs assessment for pediatric residents performing acute sexual abuse evaluations in an emergency department setting, thus laying groundwork for an adaptable curriculum. The objective was to explore pediatric resident training, knowledge, confidence, expectations, learning needs, and educational goals. PARTICIPANTS AND SETTING/METHODS:We conducted a qualitative exploratory study of pediatric residents, faculty, and program directors at two academic health centers in New York City. METHODS:Using purposive and convenience sampling, we conducted focus groups and semi-structured interviews until saturation of ideas was achieved. Through an iterative process using constructivist grounded theory, themes were organized into a curricular model. RESULTS:We conducted 3 resident focus groups (n = 21) and 7 interviews with emergency medicine, pediatric, and child abuse faculty. Themes emerged in three categories: barriers (e.g., knowledge deficits), facilitators (e.g., pre-learning), and educational goals. Despite recognizing the importance and increased availability of subspecialists, participants supported gradual autonomy for pediatric residents in the evaluation of suspected sexual abuse, with a goal of independent competency in history and examination skills, and supervised competency of forensic evidence collection. CONCLUSIONS:Our data support a multimodal, blended curriculum for the acute sexual abuse evaluation, including: (1) asynchronous pre-learning; (2) live workshops; (3) reference tools; and (4) modeled clinical experiences. Our proposed curricular model may be utilized by a variety of frontline clinicians.

BACKGROUND:Correct interpretation of the prepubescent female genital examination is a critical skill; however, physician skill in this area is limited. OBJECTIVE:To complement the bedside learning of this examination, we developed a learning platform for the visual diagnosis of the prepubescent female genital examination and examined the amount and rate of skill acquisition. PARTICIPANTS AND SETTING/METHODS:Medical students, residents, and fellows and attendings participated in an on-line learning platform. METHODS:This was a multicenter prospective cross-sectional study. Study participants deliberately practiced 158 prepubescent female genital examination cases hosted on a computer-based learning and assessment platform. Participants assigned the case normal or abnormal; if abnormal, they identified the location of the abnormality and the specific diagnosis. Participants received feedback after every case. RESULTS:We enrolled 107 participants (26 students, 31 residents, 24 fellows and 26 attendings). Accuracy (95 % CI) increased by 10.3 % (7.8, 12.8), Cohen's d-effect size of 1.17 (1.14, 1.19). The change in specificity was +16.8 (14.1, 19.5) and sensitivity +2.4 (-0.9, 5.6). It took a mean (SD) 46.3 (32.2) minutes to complete cases. There was no difference between learner types with respect to initial (p = 0.2) or final accuracy (p = 0.4) scores. CONCLUSIONS:This study's learning intervention led to effective and feasible skill improvement. However, while participants improved significantly with normal cases, which has relevance in reducing unnecessary referrals to child protection teams, learning gains were not as evident in abnormal cases. All levels of learners demonstrated a similar performance, emphasizing the need for this education even among experienced clinicians.

The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.