Faculty Bibliography

The Clinical Emergency Data Registry (CEDR) is a qualified clinical data registry that collects data from participating emergency departments (EDs) in the United States for quality measurement, improvement, and reporting purposes. This article aims to provide an overview of the data collection and validation process, describe the existing data structure and elements, and explain the potential opportunities and limitations for ongoing and future research use. CEDR data are primarily collected for quality reporting purposes and are obtained from diverse sources, including electronic health records and billing data that are de-identified and stored in a secure, centralized database. The CEDR data structure is organized around clinical episodes, which contain multiple data elements that are standardized using common data elements and are mapped to established terminologies to enable interoperability and data sharing. The data elements include patient demographics, clinical characteristics, diagnostic and treatment procedures, and outcomes. Key limitations include the limited generalizability due to the selective nature of participating EDs and the limited validation and completeness of data elements not currently used for quality reporting purposes, including demographic data. Nonetheless, CEDR holds great potential for ongoing and future research in emergency medicine due to its large-volume, longitudinal, near real-time, clinical data. In 2021, the American College of Emergency Physicians authorized the transition from CEDR to the Emergency Medicine Data Institute, which will catalyze investments in improved data quality and completeness for research to advance emergency care.

IMPORTANCE/UNASSIGNED:Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR. OBJECTIVE/UNASSIGNED:Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates. METHODS/UNASSIGNED:Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation. RESULTS/UNASSIGNED:For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date. CONCLUSIONS/UNASSIGNED:In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

BACKGROUND:In 2011, the American Board of Medical Specialties established clinical informatics (CI) as a subspecialty in medicine, jointly administered by the American Board of Pathology and the American Board of Preventive Medicine. Subsequently, many institutions created CI fellowship training programs to meet the growing need for informaticists. Although many programs share similar features, there is considerable variation in program funding and administrative structures. OBJECTIVES:The aim of our study was to characterize CI fellowship program features, including governance structures, funding sources, and expenses. METHODS:We created a cross-sectional online REDCap survey with 44 items requesting information on program administration, fellows, administrative support, funding sources, and expenses. We surveyed program directors of programs accredited by the Accreditation Council for Graduate Medical Education between 2014 and 2021. RESULTS:We invited 54 program directors, of which 41 (76%) completed the survey. The average administrative support received was $27,732/year. Most programs (85.4%) were accredited to have two or more fellows per year. Programs were administratively housed under six departments: Internal Medicine (17; 41.5%), Pediatrics (7; 17.1%), Pathology (6; 14.6%), Family Medicine (6; 14.6%), Emergency Medicine (4; 9.8%), and Anesthesiology (1; 2.4%). Funding sources for CI fellowship program directors included: hospital or health systems (28.3%), clinical departments (28.3%), graduate medical education office (13.2%), biomedical informatics department (9.4%), hospital information technology (9.4%), research and grants (7.5%), and other sources (3.8%) that included philanthropy and external entities. CONCLUSION:CI fellowships have been established in leading academic and community health care systems across the country. Due to their unique training requirements, these programs require significant resources for education, administration, and recruitment. There continues to be considerable heterogeneity in funding models between programs. Our survey findings reinforce the need for reformed federal funding models for informatics practice and training.

INTRODUCTION/UNASSIGNED:We sought to describe the demographic characteristics, clinical features, and outcomes of a cohort of patients who presented to our emergency departments with mpox (formerly known as monkeypox) infection between May 1-August 1, 2022. CASE SERIES/UNASSIGNED:We identified 145 patients tested for mpox, of whom 79 were positive. All positive cases were among cisgender men, and the majority (92%) were among men who have sex with men. A large number of patients (39%) were human immunodeficiency virus (HIV) positive. There was wide variation in emergency department (ED) length of stay (range 2-16 hours, median 4 hours) and test turnaround time (range 1-11 days, median 4 days). Most patients (95%) were discharged, although a substantial proportion (22%) had a return visit within 30 days, and 28% ultimately received tecrovirimat. CONCLUSION/UNASSIGNED:Patients who presented to our ED with mpox had similar demographic characteristics and clinical features as those described in other clinical settings during the 2022 outbreak. While there were operational challenges to the evaluation and management of these patients, demonstrated by variable lengths of stay and frequent return visits, most were able to be discharged.

Clinical Informatics (CI), a medical subspecialty since 2011, has grown from the initial four fellowship programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) in 2014 to more than 50 and counting in the present day. In parallel, the literature guiding Clinical Informatics Fellowship training and the curriculum evolved from the original core content published in 2009 to the more recent CI Subspecialty Delineation of Practice and the updated ACGME Milestones 2.0 for CI. In this paper, we outline this evolution and its impact on CIF Curricula. We then propose a framework, specific processes, and tools to standardize the design and optimize the implementation of CIF programs.

BACKGROUND:Cricothyrotomy is a critical technique for rescue of the failed airway in the emergency department (ED). Since the adoption of video laryngoscopy, the incidence of rescue surgical airways (those performed after at least one unsuccessful orotracheal or nasotracheal intubation attempt), and the circumstances where they are attempted, has not been characterized. OBJECTIVE:We report the incidence and indications for rescue surgical airways using a multicenter observational registry. METHODS:We performed a retrospective analysis of rescue surgical airways in subjects ≥14 years of age. We describe patient, clinician, airway management, and outcome variables. RESULTS:Of 19,071 subjects in NEAR, 17,720 (92.9%) were ≥14 years old with at least one initial orotracheal or nasotracheal intubation attempt, 49 received a rescue surgical airway attempt, an incidence of 2.8 cases per 1000 (0.28% [95% confidence interval 0.21 to 0.37]). The median number of airway attempts prior to rescue surgical airways was 2 (interquartile range 1, 2). Twenty-five were in trauma victims (51.0% [36.5 to 65.4]), with neck trauma being the most common traumatic indication (n = 7, 14.3% [6.4 to 27.9]). CONCLUSION:Rescue surgical airways occurred infrequently in the ED (0.28% [0.21 to 0.37]), with approximately half performed due to a trauma indication. These results may have implications for surgical airway skill acquisition, maintenance, and experience.

Patient portals are poised to transform health communication by empowering patients with rapid access to their own health data. The 21st Century Cures Act is a US federal law that, among other provisions, prevents health entities from engaging in practices that disrupt the exchange of electronic health information-a measure that may increase the usage of patient health portals. Caregiver access to patient portals, however, may lead to breaches in patient privacy and confidentiality if not managed properly through proxy accounts. We present an ethical framework that guides policy and clinical workflow development for healthcare institutions to support the best use of patient portals. Caregivers are vital members of the care team and should be supported through novel forms of health information technology (IT). Patients, however, may not want all information to be shared with their proxies so healthcare institutions must support the development and use of separate proxy accounts as opposed to using the patient's own account as well provide controls for limiting the scope of information displayed in the proxy accounts. Lastly, as socioeconomic barriers to adoption of health IT persist, healthcare providers must work to ensure multiple streams of patient communication, to prevent further propagating health inequities.

AUDIENCE/UNASSIGNED:This curriculum is designed for emergency medicine residents at all levels of training. The curriculum covers basic foundations in clinical informatics for improving patient care and outcomes, utilizing data, and leading improvements in emergency medicine. LENGTH OF CURRICULUM/UNASSIGNED:The curriculum is designed for a four-week rotation. INTRODUCTION/UNASSIGNED:The American College of Graduate Medical Education (ACGME) mandated that all Emergency Medicine (EM) residents receive specific training in the use of information technology.1,2 To our knowledge, a clinical informatics curriculum for EM residents does not exist. We propose the following standardized and reproducible educational curriculum for EM residents. EDUCATIONAL GOALS/UNASSIGNED:The aim of this curriculum is to teach informatics skills to emergency physicians to improve patient care and outcomes, utilize data, and develop projects to lead change.3 These goals will be achieved by providing a foundational informatics elective for EM residents that follows the delineation of practice for Clinical Informatics outlined by the American Medical Informatics Association (AMIA) and the American Board of Preventive Medicine (ABPM).4-6. EDUCATIONAL METHODS/UNASSIGNED:The educational strategies used in this curriculum include asynchronous learning via books, papers, videos, and websites. Residents attend administrative sessions (meetings), develop a project proposal, and participate in small group discussions.The rotation emphasizes the basic concepts surrounding clinical informatics with an emphasis on improving care delivery and outcomes, information systems, data governance and analytics, as well as leadership and professionalism. The course focuses on the practical application of these concepts, including implementation, clinical decision support, workflow analysis, privacy and security, information technology across the patient care continuum, health information exchange, data analytics, and leading change through stakeholder engagement. RESEARCH METHODS/UNASSIGNED:An initial version of the curriculum was introduced to two separate institutions and was completed by three rotating resident physicians and one rotating resident pharmacist. A brief course evaluation as well as qualitative feedback was solicited from elective participants by the course director, via email following the completion of the course, regarding the effectiveness of the course content. Learner feedback was used to influence the development of this complete curriculum. RESULTS/UNASSIGNED:The curriculum was graded by learners on a 5-point Likert scale (1=strongly disagree, 5 = strongly agree). The mean response to, "This course was a valuable use of my elective time," was 5 (sd=0). The mean response to, "I achieved the learning objectives," and "This rotation helped me understand Clinical Informatics," were both 4.75 (sd=0.5). DISCUSSION/UNASSIGNED:Overall, participants reported that the content was effective for achieving the learning objectives. During initial implementation, we found that the preliminary asynchronous learning component worked less effectively than we anticipated due to a lower volume of content. In response to this, as well as resident feedback, we added significantly more educational content.In conclusion, this model curriculum provides a structured process for an informatics rotation for the emergency medicine resident that utilizes the core competencies established by the governing bodies of the clinical informatics specialty and ACGME. TOPICS/UNASSIGNED:Clinical informatics key concepts, including definitions, fundamental terminology, history, policy and regulations, ethical considerations, clinical decision support, health information systems, data governance and analytics, process improvement, stakeholder engagement and change management.

INTRODUCTION/BACKGROUND:Early rooming triage increases patient throughput and satisfaction by rapidly assigning patients to a definitive care area, without using vital signs or detailed chart review. Despite these operational benefits, the clinical accuracy of early rooming triage is not well known. We sought to measure the accuracy of early rooming triage and uncover additional patient characteristics that can assist triage. METHODS:We conducted a single-center, retrospective population study of walk-in emergency department (ED) patients presenting to the ED via an early rooming triage system, examining triage accuracy and demographic factor correlation with higher acuity ED outcomes. RESULTS:Among all patients included from the three-year study period (N = 238,457), early rooming triage was highly sensitive (0.89) and less specific (0.61) for predicting which patients would have a severe outcome in the ED. Patients triaged to the lowest acuity area of the ED experienced severe outcomes in 4.39% of cases, while patients triaged to the highest acuity area of the ED experienced severe outcomes in 65.9% of cases. An age of greater than 43 years (odds ratio [OR] 3.48, 95% confidence interval: 3.40, 3.57) or patient's home address farther from the ED ([OR] 2.23 to 3.08) were highly correlated with severe outcomes. Multivariable models incorporating triage team judgment were robust for predicting severe outcomes at triage, with an area under the receiver operating characteristic of 0.82. CONCLUSION/CONCLUSIONS:Early rooming workflows are appropriately sensitive for ED triage. Consideration of demographic factors, automated or otherwise, can augment ED processes to provide optimal triage.