Faculty Bibliography

Statement of Problem Or Question (One Sentence): Underutilization of clinical prediction rules and poor uptake of provider-oriented clinical decision support (CDS) has contributed to overuse of antibiotics for sore throat. Objectives of Program/Intervention (No More Than Three Objectives): 1. Adapt CDS for registered nurses (RNs) to evaluate and treat patients with sore throat 2. Demonstrate the feasibility of RN visits using CDS to evaluate and treat patients with sore throat Description of Program/Intervention, Including Organizational Context (E.G. Inpatient Vs. Outpatient, Practice or Community Characteristics): We performed a 12-week pilot study to evaluate the feasibility of RN visits using an integrated clinical prediction rule (iCPR) tool to determine patient risk for strep throat and provide appropriate treatment at a family medicine clinic in a Midwest academic healthcare system. iCPR, originally developed for use by primary care physicians (PCPS), includes a risk calculator using Centor strep throat criteria and ordersets based on patient's risk for strep throat: education for low-risk, testing for intermediate-risk, and testing or antibiotics for high-risk. To adapt the process for RN visits, we developed triage protocols so appropriate patients received nurse visits, very low risk received education and more complex patients received provider visits. No major changes were made to the risk calculator or ordersets. Four RNs, with 2-24 years of experience, received a 10-minute online training session on sore throat evaluation followed by a 45-minute in-person training on physical examination and iCPR use. RNs triaged patients by phone and conducted RN visits using iCPR and following orderset recommendations. RNs could transition to a PCP visit if they were uncomfortable evaluating the patient. Measures of Success (DISCUSS QUALITATIVE AND/OR QUANTITATIVE METRICS WHICH WILL BE USED TO EVALUATE PROGRAM/INTERVENTION): Electronic health record data was used to determine the number of nurse visits, frequency of tool use and antibiotic and diagnostic test ordering. RNs completed a self-efficacy survey prior to training and 8-weeks after implementation. At 12 weeks, we interviewed RNs to understand barriers and facilitators to using the tool. Findings To Date (It Is Not Sufficient To State Findings Will Be Discussed): 162 triage calls for sore throat resulted in 77(48%) patients with RN-only visits, 45(28%) with provider visits, 38(23%) with no visit. Only 2 RN visits (< 3%) converted to provider visit due to patient complexity. RNs completed the risk calculator for 99% of visits and followed recommendations in all cases except for ordering antibiotics in 1 high-risk patient with a negative rapid strep. RN confidence in their ability to evaluate and treat a patient with sore throat was 85 (SD 5.8) (0 cannot do at all; 100 highly certain I can do) prior to training and 97.5 (SD 5.0) at 8-weeks. RNs felt the tool decreased provider visits and strep testing in patients. RN's also felt that the tool increased patient and RN satisfaction. Key Lessons For Dissemination (What Can Others Take Away For Implementation To Their Practice Or Community?): This pilot study demonstrates that RNs can use CDS to appropriately triage, evaluate and treat acute low-complexity sore throat patients. Implementation of an RN-driven iCPR tool shows promise to reduce inappropriate antibiotic prescribing and represents a potential model for expanding RN practice using CDS

Background: Clinical decision support (CDS) tools which incorporate clinical prediction rules (CPRs) have the potential to successfully deliver accurate information and guide decision-making at the point of care. Our previously validated integrated clinical prediction rule (iCPR) was designed to guide evidence-based treatment within an electronic health record for streptococcal pharyngitis and pneumonia based on chief complaints of sore throat, cough or upper respiratory infection. In initial testing at a single site, it resulted in high provider tool adoption (58%) and decreased antibiotic prescribing rates (35%) for acute respiratory infections. Our objective for this study was to assess the impact of this tool when adapted and implemented in diverse primary care settings.
Method(s): This was a randomized controlled trial including 33 primary care practices at two large academic health systems in Wisconsin and Utah. Between October 2015 and June 2018 providers in the intervention group were prompted to complete either Centor Score or Heckerling Rule for Pneumonia based onthe chief complaint of the patient encounter. EHR data on provider and patient demographics, tool use rates, and antibiotic order rates from 541 providers and 100,573 monitored patient encounters were collected for analysis. Risk ratios, CIs, and P values are calculated from a generalized estimating equation log-binomial model adjusting for clustering of orders or visits by provider and using robust standard error estimators.
Result(s): The tool was triggered 42,126 times among 214 intervention providers and was completed in 6.9% of eligible visits. The intervention and control groups prescribed antibiotics in 35% and 36% of visits respectively and were not significantly different. There were no differences in rates for rapid streptococcal test or chest X-ray orders between groups (Strep: relative risk, 1.0; P=.11; Pneumonia: relative risk, 1.8; P=.64).
Conclusion(s): In diverse primary care settings, the tool was not effective at reducing unnecessary antibiotic prescription and diagnostic testing. This outcome was possibly driven by low overall use of CDS tools highlighting the growing impact of " alert fatigue" and the need for new approaches to enhance provider engagement with CDS tools. New strategies for reducing the persistently high rates of inappropriate antibiotic prescribing for acute respiratory infections are needed. Novel approaches in future studies are necessary for reducing barriers to CDS tools in order to increase use and engagement

Background: Older adults with diabetes continue to be overtreated despite current guidelines recommending less aggressive target A1c levels based on life expectancy. The suboptimal management of this vulnerable population could be due to physicians having conflicting beliefs regarding this guideline or simply lacking awareness, and changing these behaviors is challenging. Clinical decision support (CDS) within the electronic health record (EHR) has the potential to address this issue, but effectiveness is undermined by alert fatigue and poor workflow integration. Incorporating behavioral economics into CDS tools is an innovative approach to improve adherence to these guidelines while reducing physician burden, and offers the promise of improving care in this population.
Method(s): We applied a systematic, user-centered approach to incorporate behavioral economic " nudges" into a CDS module and performed user testing in six pilot primary care practices in a large academic medical center. To build the nudges, we conducted: (1) semi-structured interviews with key informants (n=8); (2) a two-hour design thinking workshop to derive and refine initial module ideas; and (3) semi-structured group interviews at each site with clinic leaders and clinicians to elicit feedback on the module components. Clinicians were observed using the module in practice; detailed field notes were collected and summarized by module idea and usability theme for rapid iteration and refinement. Frequency of firing and user action taken were assessed in the first month of implementation via EHR reporting to confirm that module components and reporting were working as expected, and to assess utilization.
Result(s): Insights from key stakeholder and clinician group interviews identified the refill protocol, inbasket lab result, and medication preference list as candidate EHR CDS targets for the module. A new EHR navigator section notification and peer comparison message, derived from the design workshop, were also prototyped and produced. User feedback from site visits confirmed compatibility with clinical workflows, and contributed to refinement of design and content. The initial prototypes were first piloted at two sites, refined, and then activated at an additional four additional sites. Preliminary Results for the six clinics indicate that over approximately 31 weeks: 1) the navigator alert fired 1047 times for 53 unique clinicians, and 2) the refill protocol alert fired 421 times for 53 unique clinicians. Reports for the other " nudges" are in development.
Conclusion(s): Integrating behavioral economic nudges into the EHR is a promising approach to enhancing guideline awareness and adherence for older adults with diabetes. This novel pilot will demonstrate the initial feasibility and preliminary efficacy of this strategy and determine if a full-scale effectiveness trial is warranted

BACKGROUND:Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. OBJECTIVE:This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. METHODS:We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to "think aloud" while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. RESULTS:A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. CONCLUSIONS:Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

BACKGROUND:Potential of the electronic health records (EHR) and clinical decision support (CDS) systems to improve the practice of medicine has been tempered by poor design and the resulting burden they place on providers. CDS is rarely tested in the real clinical environment. As a result, many tools are hard to use, placing strain on providers and resulting in low adoption rates. The existing CDS usability literature relies primarily on expert opinion and provider feedback via survey. This is the first study to evaluate CDS usability and the provider-computer-patient interaction with complex CDS in the real clinical environment. OBJECTIVE:This study aimed to further understand the barriers and facilitators of meaningful CDS usage within a real clinical context. METHODS:This qualitative observational study was conducted with 3 primary care providers during 6 patient care sessions. In patients with the chief complaint of sore throat, a CDS tool built with the Centor Score was used to stratify the risk of group A Streptococcus pharyngitis. In patients with a chief complaint of cough or upper respiratory tract infection, a CDS tool built with the Heckerling Rule was used to stratify the risk of pneumonia. During usability testing, all human-computer interactions, including audio and continuous screen capture, were recorded using the Camtasia software. Participants' comments and interactions with the tool during clinical sessions and participant comments during a postsession brief interview were placed into coding categories and analyzed for generalizable themes. RESULTS:In the 6 encounters observed, primary care providers toggled between addressing either the computer or the patient during the visit. Minimal time was spent listening to the patient without engaging the EHR. Participants mostly used the CDS tool with the patient, asking questions to populate the calculator and discussing the results of the risk assessment; they reported the ability to do this as the major benefit of the tool. All providers were interrupted during their use of the CDS tool by the need to refer to other sections of the chart. In half of the visits, patients' clinical symptoms challenged the applicability of the tool to calculate the risk of bacterial infection. Primary care providers rarely used the incorporated incentives for CDS usage, including progress notes and patient instructions. CONCLUSIONS:Live usability testing of these CDS tools generated insights about their role in the patient-provider interaction. CDS may contribute to the interaction by being simultaneously viewed by the provider and patient. CDS can improve usability and lessen the strain it places on providers by being short, flexible, and customizable to unique provider workflow. A useful component of CDS is being as widely applicable as possible and ensuring that its functions represent the fastest way to perform a particular task.

OBJECTIVE:We conducted pre-deployment usability testing and semi-structured group interviews at 6 months post-deployment with 75 providers at 14 intervention clinics across the two sites to collect user feedback. Qualitative data analysis is bifurcated into immediate and delayed stages; we reported on immediate-stage findings from real-time field notes used to generate a set of rapid, pragmatic recommendations for iterative refinement. Monthly utilization rates were calculated and examined over 12 months. RESULTS:We hypothesized a well-validated, user-centered clinical decision support tool would lead to relatively high adoption rates. Then 6 months post-deployment, integrated clinical prediction rule study tool utilization rates were substantially lower than anticipated based on the original integrated clinical prediction rule study trial (68%) at 17% (Health System A) and 5% (Health System B). User feedback at 6 months resulted in recommendations for tool refinement, which were incorporated when possible into tool design; however, utilization rates at 12 months post-deployment remained low at 14% and 4% respectively. DISCUSSION/CONCLUSIONS:Although valuable, findings demonstrate the limitations of a user-centered approach given the complexity of clinical decision support. CONCLUSION/CONCLUSIONS:Strategies for addressing persistent external factors impacting clinical decision support adoption should be considered in addition to the user-centered design and implementation of clinical decision support.

Background: The burden of mental, neurological, and substance (MNS) disorders is greater in low- and middle-income countries (LMICs). The rapid growth of digital health (i.e., eHealth) approaches offer new solutions for transforming pediatric mental health services and have the potential to address multiple resource and system barriers. However, little work has been done in applying eHealth to promote young children's mental health in LMICs. It is also not clear how eHealth has been and might be applied to translating existing evidence-based practices/strategies (EBPs) to enable broader access to child mental health interventions and services. Methods: A scoping review was conducted to summarize current eHealth applications and evidence in child mental health. The review focuses on 1) providing an overview of existing eHealth applications, research methods, and effectiveness evidence in child mental health promotion (focused on children of 0-12 years of age) across diverse service contexts; and 2) drawing lessons learned from the existing research about eHealth design strategies and usability data in order to inform future eHealth design in LMICs. Results: Thirty-two (32) articles fitting our inclusion criteria were reviewed. The child mental health eHealth studies were grouped into three areas: i) eHealth interventions targeting families that promote child and family wellbeing; ii) eHealth for improving school mental health services (e.g., promote school staff's knowledge and management skills); and iii) eHealth for improving behavioral health care in the pediatric care system (e.g., promote use of integrated patient-portal and electronic decision support systems). Most eHealth studies have reported positive impacts. Although most pediatric eHealth studies were conducted in high-income countries, many eHealth design strategies can be adapted and modified to fit LMIC contexts. Most user-engagement strategies identified from high-income countries are also relevant for populations in LMICs. Conclusions: This review synthesizes patterns of eHealth use across a spectrum of individual/family and system level of eHealth interventions that can be applied to promote child mental health and strengthen mental health service systems. This review also summarizes critical lessons to guide future eHealth design and delivery models in LMICs. However, more research in testing combinations of eHealth strategies in LMICs is needed.

Background/UNASSIGNED:Current guidelines recommend less aggressive target hemoglobin A1c (HbA1c) levels based on older age and lower life expectancy for older adults with diabetes. The effectiveness of electronic health record (EHR) clinical decision support (CDS) in promoting guideline adherence is undermined by alert fatigue and poor workflow integration. Integrating behavioral economics (BE) and CDS tools is a novel approach to improving adherence to guidelines while minimizing clinician burden. Methods/UNASSIGNED: = 8), (2) a 2-h, design-thinking workshop to derive and refine initial module ideas, and (3) semi-structured group interviews at each site with clinic leaders and clinicians to elicit feedback on three proposed nudge module components (navigator section, inbasket refill protocol, medication preference list). Detailed field notes will be summarized by module idea and usability theme for rapid iteration. Frequency of firing and user action taken will be assessed in the first month of implementation via EHR reporting to confirm that module components and related reporting are working as expected as well as assess utilization. To assess the utilization and feasibility of the new tools and generate estimates of clinician compliance with the Choosing Wisely guideline for diabetes management in older adults, a 6-month, single-arm pilot study of the BE-EHR module will be conducted in six outpatient primary care clinics. Discussion/UNASSIGNED:We hypothesize that a low burden, user-centered approach to design will yield a BE-driven, CDS module with relatively high utilization by clinicians. The resulting module will establish a platform for exploring the ability of BE concepts embedded within the EHR to affect guideline adherence for other use cases.