Faculty Bibliography

PURPOSE/OBJECTIVE:To develop natural language processing (NLP) to identify incidental lung nodules (ILNs) in radiology reports for assessment of management recommendations. METHOD AND MATERIALS/METHODS:We searched the electronic health records for patients who underwent chest CT during 2014 and 2017, before and after implementation of a department-wide dictation macro of the Fleischner Society recommendations. We randomly selected 950 unstructured chest CT reports and reviewed manually for ILNs. An NLP tool was trained and validated against the manually reviewed set, for the task of automated detection of ILNs with exclusion of previously known or definitively benign nodules. For ILNs found in the training and validation sets, we assessed whether reported management recommendations agreed with Fleischner Society guidelines. The guideline concordance of management recommendations was compared between 2014 and 2017. RESULTS:The NLP tool identified ILNs with sensitivity and specificity of 91.1% and 82.2%, respectively, in the validation set. Positive and negative predictive values were 59.7% and 97.0%. In reports of ILNs in the training and validation sets before versus after introduction of a Fleischner reporting macro, there was no difference in the proportion of reports with ILNs (108 of 500 [21.6%] versus 101 of 450 [22.4%]; P = .8), or in the proportion of reports with ILNs containing follow-up recommendations (75 of 108 [69.4%] versus 80 of 101 [79.2%]; P = .2]. Rates of recommendation guideline concordance were not significantly different before and after implementation of the standardized macro (52 of 75 [69.3%] versus 60 of 80 [75.0%]; P = .43). CONCLUSION/CONCLUSIONS:NLP reliably automates identification of ILNs in unstructured reports, pertinent to quality improvement efforts for ILN management.

PURPOSE/OBJECTIVE:The aim of this study was to determine rates of and possible reasons for guideline-discordant ordering of CT pulmonary angiography for the evaluation of suspected pulmonary embolism (PE) in the emergency department. METHODS:A retrospective review was performed of 212 consecutive encounters (January 6, 2016, to February 25, 2016) with 208 unique patients in the emergency department that resulted in CT pulmonary angiography orders. For each encounter, the revised Geneva score and two versions of the Wells criteria were calculated. Each encounter was then classified using a two-tiered risk stratification method (PE unlikely versus PE likely). Finally, the rate of and possible explanations for guideline-discordant ordering were assessed via in-depth chart review. RESULTS:The frequency of guideline-discordant studies ranged from 53 (25%) to 79 (37%), depending on the scoring system used; 46 (22%) of which were guideline discordant under all three scoring systems. Of these, 18 (39%) had at least one patient-specific factor associated with increased risk for PE but not included in the risk stratification scores (eg, travel, thrombophilia). CONCLUSIONS:Many of the guideline-discordant orders were placed for patients who presented with evidence-based risk factors for PE that are not included in the risk stratification scores. Therefore, guideline-discordant ordering may indicate that in the presence of these factors, the assessment of risk made by current scoring systems may not align with clinical suspicion.

BACKGROUND:The assessment of clinical guideline adherence for the evaluation of pulmonary embolism (PE) via computed tomography pulmonary angiography (CTPA) currently requires either labor-intensive, retrospective chart review or prospective collection of PE risk scores at the time of CTPA order. The recording of clinical data in a structured manner in the electronic health record (EHR) may make it possible to automate the calculation of a patient's PE risk classification and determine whether the CTPA order was guideline concordant. OBJECTIVES/OBJECTIVE:The objective of this study was to measure the performance of automated, structured-data-only versions of the Wells and revised Geneva risk scores in emergency department encounters during which a CTPA was ordered. The hypothesis was that such an automated method would classify a patient's PE risk with high accuracy compared to manual chart review. METHODS:We developed automated, structured-data-only versions of the Wells and revised Geneva risk scores to classify 212 emergency department (ED) encounters during which a CTPA was performed as "PE Likely" or "PE Unlikely." We then combined these classifications with D-dimer ordering data to assess each encounter as guideline concordant or discordant. The accuracy of these automated classifications and assessments of guideline concordance were determined by comparing them to classifications and concordance based on the complete Wells and revised Geneva scores derived via abstractor manual chart review. RESULTS:The automatically derived Wells and revised Geneva risk classifications were 91.5% and 92% accurate compared to the manually determined classifications, respectively. There was no statistically significant difference between guideline adherence calculated by the automated scores as compared to manual chart review (Wells: 70.8 vs. 75%, p = 0.33 | Revised Geneva: 65.6 vs. 66%, p = 0.92). CONCLUSION/CONCLUSIONS:The Wells and revised Geneva score risk classifications can be approximated with high accuracy using automated extraction of structured EHR data elements in patients who received a CTPA. Combining these automated scores with D-dimer ordering data allows for the automated assessment of clinical guideline adherence for CTPA ordering in the emergency department, without the burden of manual chart review.

BACKGROUND: Testing for venous thromboembolism (VTE) is associated with cost and risk to patients (e.g. radiation). To assess the appropriateness of imaging utilization at the provider level, it is important to know that provider's diagnostic yield (percentage of tests positive for the diagnostic entity of interest). However, determining diagnostic yield typically requires either time-consuming, manual review of radiology reports or the use of complex and/or proprietary natural language processing software. OBJECTIVES: The objectives of this study were twofold: 1) to develop and implement a simple, user-configurable, and open-source natural language processing tool to classify radiology reports with high accuracy and 2) to use the results of the tool to design a provider-specific VTE imaging dashboard, consisting of both utilization rate and diagnostic yield. METHODS: Two physicians reviewed a training set of 400 lower extremity ultrasound (UTZ) and computed tomography pulmonary angiogram (CTPA) reports to understand the language used in VTE-positive and VTE-negative reports. The insights from this review informed the arguments to the five modifiable parameters of the NLP tool. A validation set of 2,000 studies was then independently classified by the reviewers and by the tool; the classifications were compared and the performance of the tool was calculated. RESULTS: The tool was highly accurate in classifying the presence and absence of VTE for both the UTZ (sensitivity 95.7%; 95% CI 91.5-99.8, specificity 100%; 95% CI 100-100) and CTPA reports (sensitivity 97.1%; 95% CI 94.3-99.9, specificity 98.6%; 95% CI 97.8-99.4). The diagnostic yield was then calculated at the individual provider level and the imaging dashboard was created. CONCLUSIONS: We have created a novel NLP tool designed for users without a background in computer programming, which has been used to classify venous thromboembolism reports with a high degree of accuracy. The tool is open-source and available for download at http://iturrate.com/simpleNLP. Results obtained using this tool can be applied to enhance quality by presenting information about utilization and yield to providers via an imaging dashboard.

Given the inequalities in the distribution of disease burden, geographically detailed methods of disease surveillance are needed to identify local hot spots of chronic disease. However, few data sources include the patient-level addresses needed to perform these studies. Given that individual hospitals would have access to this geographically granular data, this study assessed the reliability of estimating chronic disease prevalence using emergency department surveillance at specific hospitals. Neighborhood-level diabetes, hypertension, and asthma prevalence were estimated using emergency claims data from each individual hospital in New York City from 2009-2012. Estimates were compared to prevalence obtained from a traditional health survey. A multivariable analysis also was performed to identify which individual hospitals were more accurate at estimating citywide disease prevalence. Among 52 hospitals, variation was found in the accuracy of disease prevalence estimates using emergency department surveillance. Estimates at some hospitals, such as NYU Langone Medical Center, had strong correlations for all diseases studied (diabetes: 0.81, hypertension: 0.84, and asthma: 0.84). Hospitals with patient populations geographically distributed throughout New York City had better accuracy in estimating citywide disease prevalence. For diabetes and hypertension, hospitals with racial/ethnic patient distributions similar to Census estimates and higher fidelity of diagnosis coding also had more accurate prevalence estimates. This study demonstrated how citywide chronic disease surveillance can be performed using emergency data from specific sentinel hospitals. The findings may provide an alternative means of mapping chronic disease burden by using existing data, which may be critical in regions without resources for geographically detailed health surveillance.

Background: Multiple models have been developed to predict hospital admission for patients presenting to the ED. However, these tools suffer from multiple limitations including reliance on manual data entry (e.g. ED arrival mechanism), multiple types of data, and data that are not completely generalizable across institutions (e.g. triage score). An ideal solution would produce a disposition score that requires no data entry, employs variables already captured by all EDs, and provides a score far enough in advance to expedite admission processes. Objectives: Evaluate the discriminatory power of machine learning algorithms for predicting hospital admission at two hours of ED arrival through the sole use of the medication administration record (MAR). Methods: Our dataset included 27,757 encounters (26% admitted) from January 2013 to September 2014 and 2,109 medications encoded to RxNorm CUI numbers using MedEx. We included all medications in the MAR, including those given during prior ED visits. We employed classic and state[[Unsupported Character - Codename]]of [[Unsupported Character - Codename]]the[[Unsupported Character - Codename]]art classifiers including logistic regression, naive bayes, regularized logistic regression, classification and regression trees (CART), and linear support vector machine (SVM) with penalty parameter C. In all cases, we split the dataset into a training, validation, and test set. We used the validation set to optimize any parameters of the learning algorithm and used the test set to calculate performances. We employed 5[[Unsupported Character - Codename]]fold cross validation and reported AUC performances averaged across 5 folds. Results: The models performed with AUCs of 0.85 for linear SVM with penalty parameter C (95%CI 0.84-0.86), 0.83 for CART (95%CI 0.82-0.84), 0.79 for regularized logistic regression (95%CI 0.78-0.80), 0.70 for Naive Bayes (95%CI 0.69-0.72), and 0.68 for logistic regression (95%CI 0.67-0.69). Conclusion: MAR data is sufficient to reliably predict hospital admission two hours into the ED stay. Our models perform similarly to those from prior studies, but with the advantages of only requiring a single type of data and being highly generalizable to other institutions; MAR data is objective, does not require manual data entry, and is universally available across EDs

Data fragmentation within electronic health records causes gaps in the information readily available to clinicians. We investigated the information needs of emergency medicine clinicians in order to design an electronic dashboard to fill information gaps in the emergency department. An online survey was distributed to all emergency medicine physicians at a large, urban academic medical center. The survey response rate was 48% (52/109). The clinical information items reported to be most helpful while caring for patients in the emergency department were vital signs, electrocardiogram (ECG) reports, previous discharge summaries, and previous lab results. Brief structured interviews were also conducted with 18 clinicians during their shifts in the emergency department. From the interviews, three themes emerged: 1) difficulty accessing vital signs, 2) difficulty accessing point-of-care tests, and 3) difficulty comparing the current ECG with the previous ECG. An emergency medicine clinical dashboard was developed to address these difficulties.

Activation of TLR by ssRNA after FcgammaR-mediated phagocytosis of immune complexes (IC) may be relevant in autoimmune-associated congenital heart block (CHB) where the obligate factor is a maternal anti-SSA/Ro Ab and the fetal factors, protein/RNA on an apoptotic cardiocyte and infiltrating macrophages. This study addressed the hypothesis that Ro60-associated ssRNAs link macrophage activation to fibrosis via TLR engagement. Both macrophage transfection with noncoding ssRNA that bind Ro60 and an IC generated by incubation of Ro60-ssRNA with an IgG fraction from a CHB mother or affinity purified anti-Ro60 significantly increased TNF-alpha secretion, an effect not observed using control RNAs or normal IgG. Dependence on TLR was supported by the significant inhibition of TNF-alpha release by IRS661 and chloroquine. The requirement for FcgammaRIIIa-mediated delivery was provided by inhibition with an anti-CD16a Ab. Fibrosis markers were noticeably increased in fetal cardiac fibroblasts after incubation with supernatants generated from macrophages transfected with ssRNA or incubated with the IC. Supernatants generated from macrophages with ssRNA in the presence of IRS661 or chloroquine did not cause fibrosis. In a CHB heart, but not a healthy heart, TLR7 immunostaining was localized to a region near the atrioventricular groove at a site enriched in mononuclear cells and fibrosis. These data support a novel injury model in CHB, whereby endogenous ligand, Ro60-associated ssRNA, forges a nexus between TLR ligation and fibrosis instigated by binding of anti-Ro Abs to the target protein likely accessible via apoptosis