Faculty Bibliography

BACKGROUND:Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools. OBJECTIVE:This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care. METHODS:A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior. RESULTS:To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images. CONCLUSIONS:These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS. TRIAL REGISTRATION/BACKGROUND:Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

BACKGROUND:The transformation of health care during COVID-19, with the rapid expansion of telemedicine visits, presents new challenges to chronic care and preventive health providers. Clinical decision support (CDS) is critically important to chronic care providers, and CDS malfunction is common during times of change. It is essential to regularly reassess an organization's ambulatory CDS program to maintain care quality. This is especially true after an immense change, like the COVID-19 telemedicine expansion. OBJECTIVE:Our objective is to reassess the ambulatory CDS program at a large academic medical center in light of telemedicine's expansion in response to the COVID-19 pandemic. METHODS:Our clinical informatics team devised a practical framework for an intrapandemic ambulatory CDS assessment focused on the impact of the telemedicine expansion. This assessment began with a quantitative analysis comparing CDS alert performance in the context of in-person and telemedicine visits. Board-certified physician informaticists then completed a formal workflow review of alerts with inferior performance in telemedicine visits. Informaticists then reported on themes and optimization opportunities through the existing CDS governance structure. RESULTS:Our assessment revealed that 10 of our top 40 alerts by volume were not firing as expected in telemedicine visits. In 3 of the top 5 alerts, providers were significantly less likely to take action in telemedicine when compared to office visits. Cumulatively, alerts in telemedicine encounters had an action taken rate of 5.3% (3257/64,938) compared to 8.3% (19,427/233,636) for office visits. Observations from a clinical informaticist workflow review included the following: (1) Telemedicine visits have different workflows than office visits. Some alerts developed for the office were not appearing at the optimal time in the telemedicine workflow. (2) Missing clinical data is a common reason for the decreased alert firing seen in telemedicine visits. (3) Remote patient monitoring and patient-reported clinical data entered through the portal could replace data collection usually completed in the office by a medical assistant or registered nurse. CONCLUSIONS:In a large academic medical center at the pandemic epicenter, an intrapandemic ambulatory CDS assessment revealed clinically significant CDS malfunctions that highlight the importance of reassessing ambulatory CDS performance after the telemedicine expansion.

Background/UNASSIGNED:Effective therapies to combat coronavirus 2019 (COVID-19) are urgently needed. Hydroxychloroquine (HCQ) has in vitro antiviral activity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but the clinical benefit of HCQ in treating COVID-19 is unclear. Randomized controlled trials are needed to determine the safety and efficacy of HCQ for the treatment of hospitalized patients with COVID-19. Methods/UNASSIGNED:We conducted a multicenter, double-blind randomized clinical trial of HCQ among patients hospitalized with laboratory-confirmed COVID-19. Subjects were randomized in a 1:1 ratio to HCQ or placebo for 5 days and followed for 30 days. The primary efficacy outcome was a severe disease progression composite end point (death, intensive care unit admission, mechanical ventilation, extracorporeal membrane oxygenation, and/or vasopressor use) at day 14. Results/UNASSIGNED: = .350). There were no significant differences in COVID-19 clinical scores, number of oxygen-free days, SARS-CoV-2 clearance, or adverse events between HCQ and placebo. HCQ was associated with a slight increase in mean corrected QT interval, an increased D-dimer, and a trend toward an increased length of stay. Conclusions/UNASSIGNED:In hospitalized patients with COVID-19, our data suggest that HCQ does not prevent severe outcomes or improve clinical scores. However, our conclusions are limited by a relatively small sample size, and larger randomized controlled trials or pooled analyses are needed.

BACKGROUND:The emergency department is a critical juncture in the trajectory of care of patients with serious, life-limiting illness. Implementation of a clinical decision support (CDS) tool automates identification of older adults who may benefit from palliative care instead of relying upon providers to identify such patients, thus improving quality of care by assisting providers with adhering to guidelines. The Primary Palliative Care for Emergency Medicine (PRIM-ER) study aims to optimize the use of the electronic health record by creating a CDS tool to identify high risk patients most likely to benefit from primary palliative care and provide point-of-care clinical recommendations. METHODS:A clinical decision support tool entitled Emergency Department Supportive Care Clinical Decision Support (Support-ED) was developed as part of an institutionally-sponsored value based medicine initiative at the Ronald O. Perelman Department of Emergency Medicine at NYU Langone Health. A multidisciplinary approach was used to develop Support-ED including: a scoping review of ED palliative care screening tools; launch of a workgroup to identify patient screening criteria and appropriate referral services; initial design and usability testing via the standard System Usability Scale questionnaire, education of the ED workforce on the Support-ED background, purpose and use, and; creation of a dashboard for monitoring and feedback. RESULTS:The scoping review identified the Palliative Care and Rapid Emergency Screening (P-CaRES) survey as a validated instrument in which to adapt and apply for the creation of the CDS tool. The multidisciplinary workshops identified two primary objectives of the CDS: to identify patients with indicators of serious life limiting illness, and to assist with referrals to services such as palliative care or social work. Additionally, the iterative design process yielded three specific patient scenarios that trigger a clinical alert to fire, including: 1) when an advance care planning document was present, 2) when a patient had a previous disposition to hospice, and 3) when historical and/or current clinical data points identify a serious life-limiting illness without an advance care planning document present. Monitoring and feedback indicated a need for several modifications to improve CDS functionality. CONCLUSIONS:CDS can be an effective tool in the implementation of primary palliative care quality improvement best practices. Health systems should thoughtfully consider tailoring their CDSs in order to adapt to their unique workflows and environments. The findings of this research can assist health systems in effectively integrating a primary palliative care CDS system seamlessly into their processes of care. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov Identifier: NCT03424109. Registered 6 February 2018, Grant Number: AT009844-01.

BACKGROUND: Enhanced Recovery after Surgery (ERAS) pathways have been shown to reduce length of stay, but there have been limited evaluations of novel electronic health record (EHR)-based pathways. Compliance with ERAS in real-world settings has been problematic. OBJECTIVE: This article evaluates a novel ERAS electronic pathway (E-Pathway) activity integrated with the EHR for patients undergoing elective colorectal surgery. METHODS: We performed a retrospective cohort study of surgical patients age ≥ 18 years hospitalized from March 1, 2013 to August 31, 2016. The primary cohort consisted of patients admitted for elective colon surgery. We also studied a control group of patients undergoing other elective procedures. The E-Pathway was implemented on March 2, 2015. The primary outcome was variable costs per case. Secondary outcomes were observed to expected length of stay and 30-day readmissions. RESULTS: = 0.231) decrease in monthly costs of 0.57% (95% CI 1.51 to - 0.37%) postintervention. For the 30-day readmission rates, there were no statistically significant changes in either cohort. CONCLUSION/CONCLUSIONS: Our study is the first to report on the reduced costs after implementation of a novel sophisticated E-Pathway for ERAS. E-Pathways can be a powerful vehicle to support ERAS adoption.

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

BACKGROUND:Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions. OBJECTIVES/OBJECTIVE:To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure. METHODS:Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS. RESULTS:percentile) of 14 (5,32) alerts were triggered per hospitalization. CONCLUSIONS:A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration. CLINICALTRIALS. GOV IDENTIFIER/UNASSIGNED:NCT02858674.

Background: Clinical decision support (CDS) systems can be valuable resources in chronic disease management, but provider utilization of these tools and their integration into workflow remains suboptimal. We used a user-centered design approach to build a CDS recommending evidence-based therapy for heart failure in an inpatient setting. We implemented two versions of the CDS: an interruptive (pop-up) alert and a non-interruptive alert displayed in a provider checklist activity. In a prior study, we found that the interruptive alert was more effective than the non-interruptive alert but suffered from a high dismissal rate. The purpose of this study was to understand provider's perceptions of factors impacting CDS utilization following its implementation.
Method(s): We performed a qualitative study following implementation of two versions of a CDS at an academic medical center. We recruited providers who had either version of the CDS triggered in the prior 24 hours and obtained feedback through semi-structured interviews. Interviews were recorded, transcribed, and double-coded. We performed a constant comparative analysis of the transcripts and developed a coding scheme informed by the Five Rights of CDS combined with Proctor's outcomes for implementation research framework. We recruited participants until thematic saturation was achieved.
Result(s): Fourteen providers participated in interviews. In general, providers found the CDS triggered for appropriate patients, provided the right information to determine appropriateness of recommendations, and had good usability. At least four providers believed they were not the right person to receive the alert, reporting that they primarily participated in cross-coverage, worked in a setting where these medications were typically contraindicated, or were already fully compliant with evidence-based medications. Providers complained that the interruptive alert led to workflow disruption and that frequently they would "just need to click through all of this." Nonetheless, many providers still preferred the interruptive version of the alert because they were either: 1) unaware of the non-interruptive alert in the checklist; 2) "don't use the provider checklist" where the non-interruptive alert resided; or 3) were unaware of the provider checklist.
Conclusion(s): We found that CDS was generally found to be acceptable, although the interruptive version of the alert was limited by disruptions in workflow. The interruptive alert was ultimately more successful as providers were unaware of the existence of the non-interruptive alert. Furthermore, they infrequently used the provider checklist, a native EHR feature for non-interruptive alerts and messages. Our findings suggest that incorporating user-centered design features can lead to success of an interruptive alert. Furthermore, future efforts to implement non-interruptive alerts must incorporate approaches to increase awareness of the alert and encourage changes in workflow to monitor alert lists

BACKGROUND:Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. OBJECTIVE:This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. METHODS:We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to "think aloud" while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. RESULTS:A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. CONCLUSIONS:Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

Objective: The purpose of this study was to determine whether an electronic health record-based sepsis alert system could improve quality of care and clinical outcomes for patients with sepsis. Materials and Methods: We performed a patient-level interrupted time series study of emergency department patients with severe sepsis or septic shock between January 2013 and April 2015. The intervention, introduced in February 2014, was a system of interruptive sepsis alerts triggered by abnormal vital signs or laboratory results. Primary outcomes were length of stay (LOS) and in-hospital mortality; other outcomes included time to first lactate and blood cultures prior to antibiotics. We also assessed sensitivity, positive predictive value (PPV), and clinician response to the alerts. Results: Mean LOS for patients with sepsis decreased from 10.1 to 8.6 days ( P < .001) following alert introduction. In adjusted time series analysis, the intervention was associated with a decreased LOS of 16% (95% CI, 5%-25%; P = .007, with significance of alpha = 0.006) and no change thereafter (0%; 95% CI, -2%, 2%). The sepsis alert system had no effect on mortality or other clinical or process measures. The intervention had a sensitivity of 80.4% and a PPV of 14.6%. Discussion: Alerting based on simple laboratory and vital sign criteria was insufficient to improve sepsis outcomes. Alert fatigue due to the low PPV is likely the primary contributor to these results. Conclusion: A more sophisticated algorithm for sepsis identification is needed to improve outcomes.