Faculty Bibliography

OBJECTIVES/OBJECTIVE:To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings. MATERIALS AND METHODS/METHODS:Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale. RESULTS:For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision. CONCLUSION/CONCLUSIONS:GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

BACKGROUND:Review of emergency department (ED) revisits with admission allows the identification of improvement opportunities. Applying a health equity lens to revisits may highlight potential disparities in care transitions. Universal definitions or practicable frameworks for these assessments are lacking. The authors aimed to develop a structured methodology for this quality assurance (QA) process, with a layered equity analysis. METHODS:The authors developed a classification instrument to identify potentially preventable 72-hour returns with admission (PPRA-72), accounting for directed, unrelated, unanticipated, or disease progression returns. A second review team assessed the instrument reliability. A self-reported race/ethnicity (R/E) and language algorithm was developed to minimize uncategorizable data. Disposition distribution, return rates, and PPRA-72 classifications were analyzed for disparities using Pearson chi-square and Fisher's exact tests. RESULTS:The PPRA-72 rate was 4.8% for 2022 ED return visits requiring admission. Review teams achieved 93% agreement (κ = 0.51) for the binary determination of PPRA-72 vs. nonpreventable returns. There were significant differences between R/E and language in ED dispositions (p < 0.001), with more frequent admissions for the R/E White at the index visit and Other at the 72-hour return visit. Rates of return visits within 72 hours differed significantly by R/E (p < 0.001) but not by language (p = 0.156), with the R/E Black most frequent to have a 72-hour return. There were no differences between R/E (p = 0.446) or language (p = 0.248) in PPRA-72 rates. The initiative led to system improvements through informatics optimizations, triage protocols, provider feedback, and education. CONCLUSION/CONCLUSIONS:The authors developed a review methodology for identifying improvement opportunities across ED 72-hour returns. This QA process enabled the identification of areas of disparity, with the continuous aim to develop next steps in ensuring health equity in care transitions.

BACKGROUND AND OBJECTIVES/OBJECTIVE:Pay-for-performance (P4P) is broadly defined as financial incentives to providers for attaining prespecified quality outcomes. Providers, payers, and public officials have worked over the years to develop innovative solutions to rapidly and consistently bring new diagnostic tests and therapies to our patients. P4P has been instituted in various forms over the last 30 years. Vaccines are one of society's greatest public health innovations and vaccination programs provide a unique opportunity for P4P programs. We attempted to investigate the effect of P4P compensation model implementation on the vaccination rate. METHODS:Utilizing a systematic review and meta-analysis approach, we searched PubMed, Embase, Scopus, and Web of Science from inception to December 2018. RESULTS:Nine articles were included with poor to moderate quality. Improvements in vaccination rates after implementation of P4P were statistically significant in 8 of 9 of studies. However, due to the heterogeneity of the methods used, we could not pool the data. CONCLUSION/CONCLUSIONS:The results of this systematic review indicate that the implementation of P4P programs can increase the vaccination rate. In recent times when it has become increasingly more popular not to vaccinate, implementing P4P becomes even more important if it is shown to be an effective tool in increasing vaccination rates.