Faculty Bibliography

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

BACKGROUND:Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions. OBJECTIVES/OBJECTIVE:To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure. METHODS:Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS. RESULTS:percentile) of 14 (5,32) alerts were triggered per hospitalization. CONCLUSIONS:A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration. CLINICALTRIALS. GOV IDENTIFIER/UNASSIGNED:NCT02858674.

Background: Clinical decision support (CDS) systems can be valuable resources in chronic disease management, but provider utilization of these tools and their integration into workflow remains suboptimal. We used a user-centered design approach to build a CDS recommending evidence-based therapy for heart failure in an inpatient setting. We implemented two versions of the CDS: an interruptive (pop-up) alert and a non-interruptive alert displayed in a provider checklist activity. In a prior study, we found that the interruptive alert was more effective than the non-interruptive alert but suffered from a high dismissal rate. The purpose of this study was to understand provider's perceptions of factors impacting CDS utilization following its implementation.
Method(s): We performed a qualitative study following implementation of two versions of a CDS at an academic medical center. We recruited providers who had either version of the CDS triggered in the prior 24 hours and obtained feedback through semi-structured interviews. Interviews were recorded, transcribed, and double-coded. We performed a constant comparative analysis of the transcripts and developed a coding scheme informed by the Five Rights of CDS combined with Proctor's outcomes for implementation research framework. We recruited participants until thematic saturation was achieved.
Result(s): Fourteen providers participated in interviews. In general, providers found the CDS triggered for appropriate patients, provided the right information to determine appropriateness of recommendations, and had good usability. At least four providers believed they were not the right person to receive the alert, reporting that they primarily participated in cross-coverage, worked in a setting where these medications were typically contraindicated, or were already fully compliant with evidence-based medications. Providers complained that the interruptive alert led to workflow disruption and that frequently they would "just need to click through all of this." Nonetheless, many providers still preferred the interruptive version of the alert because they were either: 1) unaware of the non-interruptive alert in the checklist; 2) "don't use the provider checklist" where the non-interruptive alert resided; or 3) were unaware of the provider checklist.
Conclusion(s): We found that CDS was generally found to be acceptable, although the interruptive version of the alert was limited by disruptions in workflow. The interruptive alert was ultimately more successful as providers were unaware of the existence of the non-interruptive alert. Furthermore, they infrequently used the provider checklist, a native EHR feature for non-interruptive alerts and messages. Our findings suggest that incorporating user-centered design features can lead to success of an interruptive alert. Furthermore, future efforts to implement non-interruptive alerts must incorporate approaches to increase awareness of the alert and encourage changes in workflow to monitor alert lists

BACKGROUND:Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. OBJECTIVE:This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. METHODS:We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to "think aloud" while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. RESULTS:A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. CONCLUSIONS:Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

Objective: The purpose of this study was to determine whether an electronic health record-based sepsis alert system could improve quality of care and clinical outcomes for patients with sepsis. Materials and Methods: We performed a patient-level interrupted time series study of emergency department patients with severe sepsis or septic shock between January 2013 and April 2015. The intervention, introduced in February 2014, was a system of interruptive sepsis alerts triggered by abnormal vital signs or laboratory results. Primary outcomes were length of stay (LOS) and in-hospital mortality; other outcomes included time to first lactate and blood cultures prior to antibiotics. We also assessed sensitivity, positive predictive value (PPV), and clinician response to the alerts. Results: Mean LOS for patients with sepsis decreased from 10.1 to 8.6 days ( P < .001) following alert introduction. In adjusted time series analysis, the intervention was associated with a decreased LOS of 16% (95% CI, 5%-25%; P = .007, with significance of alpha = 0.006) and no change thereafter (0%; 95% CI, -2%, 2%). The sepsis alert system had no effect on mortality or other clinical or process measures. The intervention had a sensitivity of 80.4% and a PPV of 14.6%. Discussion: Alerting based on simple laboratory and vital sign criteria was insufficient to improve sepsis outcomes. Alert fatigue due to the low PPV is likely the primary contributor to these results. Conclusion: A more sophisticated algorithm for sepsis identification is needed to improve outcomes.

Background: Paper-based Early Recovery after Surgery (ERAS) path-ways have been shown to reduce length of stay, but there have been limited evaluations of electronic health record (EHR) based pathways. The objective of this study was to evaluate whether ERAS processes implemented with a novel pathway activity integrated with the EHR (E-Pathway) can reduce costs without resulting in increased 30 day readmissions. Methods: We performed a retrospective cohort study of surgical patients age>= 18 years hospitalized at an academic medical center from March 1, 2013 to August 31, 2016. The primary cohort consisted of patients admitted for elective colon surgery. We also studied a control group of patients undergoing elective procedures with similar length of stay as colon surgery (3-5 days). The E-Pathway was based on a pathway template developed by a common EHR vendor (Epic Systems, Verona, WI) with content developed by a multidisciplinary team based on ERAS principles. The E-Pathway was implemented on March 2, 2015. The primary outcome was variable costs per case. Secondary outcomes were observed to expected length of stay (O:E LOS) and 30 day readmissions to our hospital. For both groups, we performed an interrupted time series with segmented regression analysis with month being the unit of time. We used gamma regression for cost models and logistic models for the secondary outcomes. Results: We included 823 (470 and 353 in the pre-and post-intervention, respectively) colon surgery patients and 3415 (1,819 and 1,596 in the pre-and post-intervention) surgical control patients. Among the colon surgery cohort, we observed no changesin cost eitheratbaseline [-0.1% (95% CI-0.8%, 0.5%) per month] or with immediate introduction of the pathway. However, there was statistically significant (p = 0.040) decrease in costs of 1.3% (0.6% to 2.5%) per surgical encounter per month over the 18 month post intervention period. The surgical comparator group had no change in cost either at baseline or at the time of intervention and had a nonsignificant decrease in monthly costs of 0.6% (p = 0.231) post-intervention. There was statistically significant (p = 0.039) decrease in the O:E slope after the intervention of 1.49% per surgical encounter per month. The surgical comparator group had a nonsignificant (p = 0.761) increase in slope of 1.87%. For the 30 day readmission rates, there were no statistically significant changes in either the colon surgery or control groups. Conclusions: Our study is the first, to our knowledge, to report on the outcomes of a novel sophisticated E pathway integrated into an EHR. Following implementation of the E-pathway for colon surgery patients, we observed decreasing direct variable costs and O:E LOS over time. These improvements were not observed among comparable surgical patients. Consequently, as institutions continue to place increased emphasis on standardization of best practice care, E-pathways can be powerful vehicles to support those changes in the new EHR-centric care model

Background: Clinical decision support (CDS) has been shown to im-prove compliance with evidence-based care but its impact is often diminished due to issues such as poor usability, insufficient integration into workflow, and alert fatigue. Non-interruptive CDS may be less subject to alert fatigue but there has been little assessment of its usability. The purpose of this study was to perform usability testing on interruptive and non-interruptive versions of a CDS. Methods: We conducted a usability study ofa CDS tool that recommended prescribing an angiotensin converting enzyme (ACE) inhibitor for inpatients with heart failure. We developed two versions of the CDS that varied in its format: an interruptive alert, in which the CDS popped-up at the time of order entry, and a non-interruptive alert, which was displayed in a checklist section of the Electronic Health Record (EHR). We recruited inpatient providers to use both versions in a laboratory setting. We randomly assigned providers to first trigger the interruptive or non-interruptive alert. Providers were given a clinical scenario and asked to " think aloud" as they worked through the CDS; we then conducted a brief semi-structured interview about usability. We used a constant comparative analysis informed by the Five Rights of CDS framework to analyze the interviews. Inpatient providers from different disciplines were recruited until thematic saturation was reached. Results: Of 12 providers who participated in usability testing, seven used the interruptive followed by the non-interruptive CDS and five used the non-interruptive CDS initially. We categorized codes into four themes related to the Five Rights of CDS and determined some codes to be general to the CDS while others were specific to the interruptive or non-interruptive version (Table). Providers noted that the interruptive alert was readily noticed but generally impeded workflow. Providers found the non-interruptive CDS to be less annoying but had lower visibility; although they liked the ability to address the non-interruptive CDS at any time, some providers questioned whether it would ultimately be used. Conclusions: Providers expressed annoyance in working with an inter-ruptive CDS. Although the non-interruptive CDS was more appealing, providers admitted that it may not be used unless integrated with workflow. One potential solution was a combination of the two formats: supplementing a non-interruptive alert with an occasional, well-timed interruptive alert if uptake was insufficient

BACKGROUND: In 2012 The Joint Commission implemented new Tobacco Treatment (TOB) performance measures for hospitals. A study evaluated the impact of a hospital-based electronic health record (EHR) intervention on adherence to the revised TOB measures. METHODS: The study was conducted in two acute care hospitals in New York City. Data abstracted from the EHR were analyzed retrospectively from 4,871 smokers discharged between December 2012 and March 2015 to evaluate the impact of two interventions: an order set to prompt clinicians to prescribe pharmacotherapy and a nurse-delivered counseling module that automatically populated the nursing care plan for all smokers. The study estimated the relative odds of a patient being prescribed medication and/or receiving smoking cessation counseling in the intervention period compared to the baseline time period. RESULTS: There was a modest increase in medication orders (odds ratio [OR], 1.35). In contrast, rates of counseling increased 10-fold (OR, 10.54). Patients admitted through surgery were less likely to receive both counseling and medication compared with the medicine service. CONCLUSION: Hospitalization presents an important opportunity to engage smokers in treatment for primary and secondary prevention of tobacco-related illnesses. EHRs can be leveraged to facilitate integration of TOB measure requirements into routine inpatient care; however, the smaller effect on prescribing patterns suggests limitations in this approach alone in changing clinician behavior to meet this measure. The success of the nurse-focused EHR-driven intervention suggests an effective tool for integrating the cessation counseling component of the new measures and the importance of nursing's role in achieving the Joint Commission measure targets.

BACKGROUND: Electronic health record (EHR)-based clinical decision support tools (CDS) incorporate individualized data to produce patient-specific recommendations at the point-of-care. However, these tools are often limited in their effectiveness, which may be due to poor consideration of usability. The purpose of this study was to evaluate the utilization of a CDS intervention to increase prescription of Angiotensin Converting Enzyme inhibitor (ACEi) or Angiotensin Receptor Blocker (ARB) for patients with heart failure. METHODS: We performed a retrospective study of hospitalized patients with heart failure from the time of CDS implementation, 7/10/13, through 11/30/15. The CDS that we investigated offers providers an opportunity to prescribe an ACEi or ARB or report a contraindication to therapy for patients with documented heart failure. All patients with an EF < 40% who were not on an ACEi or ARB at time of discharge were included in the study. We identified the number of patients for whom the CDS triggered; of those, we categorized provider response as: dismissed, ordered an ACEi/ARB, or contraindication reported. We then performed manual chart review to identify the CDS reported contraindication and structured chart abstraction with standard guidelines to identify gold standard contraindications. We compared each CDS contraindication to gold standard contraindications to determine their accuracy. RESULTS: Out of the 618 subjects who had an EF < 40% but no ACEi or ARB at the time or discharge, 435/618 (70%) had a triggered CDS. Of these 435 subjects for who a CDS was triggered, 180 (41%) were dismissed, 225 (52%) had a contraindication response and 30 (7%) had a prescription for an ACEi/ARB therapy. Overall the accuracy of the documented CDS was 42% (Table 1). CONCLUSIONS: The CDS that we reviewed was poorly utilized and contraindications documented in the tool poorly correlated with patient clinical status reflected elsewhere in the EHR. These findings identify this CDS as a possible impedance to user workflow. One way to improve CDS tools at the point of care is through thorough usability testing and consideration of physician workflow prior to implementation. (Table Presented)

BACKGROUND: Hospital care on weekends has been associated with delays in care, reduced quality, and poor clinical outcomes. OBJECTIVE: The purpose of this study was to evaluate the impact of a weekend hospital intervention on processes of care and clinical outcomes. The multifaceted intervention included expanded weekend diagnostic services, improved weekend discharge processes, and increased physician and care management services on weekends. DESIGN AND PATIENTS: This was an interrupted time series observational study of adult non-obstetric patients hospitalized at a single academic medical center between January 2011 and January 2014. The study included 18 months prior to and 19 months following the implementation of the intervention. Data were analyzed using segmented regression analysis with adjustment for confounders. MAIN MEASURES: The primary outcome was average length of stay. Secondary outcomes included percent of patients discharged on weekends, 30-day readmission rate, and in-hospital mortality rate. KEY RESULTS: The study included 57,163 hospitalizations. Following implementation of the intervention, average length of stay decreased by 13 % (95 % CI 10-15 %) and continued to decrease by 1 % (95 % CI 1-2 %) per month as compared to the underlying time trend. The proportion of weekend discharges increased by 12 % (95 % CI 2-22 %) at the time of the intervention and continued to increase by 2 % (95 % CI 1-3 %) per month thereafter. The intervention had no impact on readmissions or mortality. During the post-implementation period, the hospital was evacuated and closed for 2 months due to damage from Hurricane Sandy, and a new hospital-wide electronic health record was introduced. The contributions of these events to our findings are not known. We observed a lower inpatient census and found differences in patient characteristics, including higher rates of Medicaid insurance and comorbidities, in the post-Hurricane Sandy period as compared to the pre-Sandy period. CONCLUSIONS: The intervention was associated with a reduction in length of stay and an increase in weekend discharges. Our longitudinal study also illuminated the challenges of evaluating the effectiveness of a large-scale intervention in a real-world hospital setting.