Faculty Bibliography

BACKGROUND:Up to 50% of antibiotic prescriptions for upper respiratory infections (URIs) are inappropriate. Clinical decision support (CDS) systems to mitigate unnecessary antibiotic prescriptions have been implemented into electronic health records, but their use by providers has been limited. OBJECTIVE:As a delegation protocol, we adapted a validated electronic health record-integrated clinical prediction rule (iCPR) CDS-based intervention for registered nurses (RNs), consisting of triage to identify patients with low-acuity URI followed by CDS-guided RN visits. It was implemented in February 2022 as a randomized controlled stepped-wedge trial in 43 primary and urgent care practices within 4 academic health systems in New York, Wisconsin, and Utah. While issues were pragmatically addressed as they arose, a systematic assessment of the barriers to implementation is needed to better understand and address these barriers. METHODS:We performed a retrospective case study, collecting quantitative and qualitative data regarding clinical workflows and triage-template use from expert interviews, study surveys, routine check-ins with practice personnel, and chart reviews over the first year of implementation of the iCPR intervention. Guided by the updated CFIR (Consolidated Framework for Implementation Research), we characterized the initial barriers to implementing a URI iCPR intervention for RNs in ambulatory care. CFIR constructs were coded as missing, neutral, weak, or strong implementation factors. RESULTS:Barriers were identified within all implementation domains. The strongest barriers were found in the outer setting, with those factors trickling down to impact the inner setting. Local conditions driven by COVID-19 served as one of the strongest barriers, impacting attitudes among practice staff and ultimately contributing to a work infrastructure characterized by staff changes, RN shortages and turnover, and competing responsibilities. Policies and laws regarding scope of practice of RNs varied by state and institutional application of those laws, with some allowing more clinical autonomy for RNs. This necessitated different study procedures at each study site to meet practice requirements, increasing innovation complexity. Similarly, institutional policies led to varying levels of compatibility with existing triage, rooming, and documentation workflows. These workflow conflicts were compounded by limited available resources, as well as an implementation climate of optional participation, few participation incentives, and thus low relative priority compared to other clinical duties. CONCLUSIONS:Both between and within health care systems, significant variability existed in workflows for patient intake and triage. Even in a relatively straightforward clinical workflow, workflow and cultural differences appreciably impacted intervention adoption. Takeaways from this study can be applied to other RN delegation protocol implementations of new and innovative CDS tools within existing workflows to support integration and improve uptake. When implementing a system-wide clinical care intervention, considerations must be made for variability in culture and workflows at the state, health system, practice, and individual levels. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04255303; https://clinicaltrials.gov/ct2/show/NCT04255303.

OBJECTIVES/OBJECTIVE:To evaluate demographic biases in diagnostic accuracy and health advice between generative artificial intelligence (AI) (ChatGPT GPT-4) and traditional symptom checkers like WebMD. MATERIALS AND METHODS/METHODS:Combination symptom and demographic vignettes were developed for 27 most common symptom complaints. Standardized prompts, written from a patient perspective, with varying demographic permutations of age, sex, and race/ethnicity were entered into ChatGPT (GPT-4) between July and August 2023. In total, 3 runs of 540 ChatGPT prompts were compared to the corresponding WebMD Symptom Checker output using a mixed-methods approach. In addition to diagnostic correctness, the associated text generated by ChatGPT was analyzed for readability (using Flesch-Kincaid Grade Level) and qualitative aspects like disclaimers and demographic tailoring. RESULTS:ChatGPT matched WebMD in 91% of diagnoses, with a 24% top diagnosis match rate. Diagnostic accuracy was not significantly different across demographic groups, including age, race/ethnicity, and sex. ChatGPT's urgent care recommendations and demographic tailoring were presented significantly more to 75-year-olds versus 25-year-olds (P < .01) but were not statistically different among race/ethnicity and sex groups. The GPT text was suitable for college students, with no significant demographic variability. DISCUSSION/CONCLUSIONS:The use of non-health-tailored generative AI, like ChatGPT, for simple symptom-checking functions provides comparable diagnostic accuracy to commercially available symptom checkers and does not demonstrate significant demographic bias in this setting. The text accompanying differential diagnoses, however, suggests demographic tailoring that could potentially introduce bias. CONCLUSION/CONCLUSIONS:These results highlight the need for continued rigorous evaluation of AI-driven medical platforms, focusing on demographic biases to ensure equitable care.

BACKGROUND:Generative artificial intelligence has the potential to revolutionize health technology product development by improving coding quality, efficiency, documentation, quality assessment and review, and troubleshooting. OBJECTIVE:This paper explores the application of a commercially available generative artificial intelligence tool (ChatGPT) to the development of a digital health behavior change intervention designed to support patient engagement in a commercial digital diabetes prevention program. METHODS:We examined the capacity, advantages, and limitations of ChatGPT to support digital product idea conceptualization, intervention content development, and the software engineering process, including software requirement generation, software design, and code production. In total, 11 evaluators, each with at least 10 years of experience in fields of study ranging from medicine and implementation science to computer science, participated in the output review process (ChatGPT vs human-generated output). All had familiarity or prior exposure to the original personalized automatic messaging system intervention. The evaluators rated the ChatGPT-produced outputs in terms of understandability, usability, novelty, relevance, completeness, and efficiency. RESULTS:Most metrics received positive scores. We identified that ChatGPT can (1) support developers to achieve high-quality products faster and (2) facilitate nontechnical communication and system understanding between technical and nontechnical team members around the development goal of rapid and easy-to-build computational solutions for medical technologies. CONCLUSIONS:ChatGPT can serve as a usable facilitator for researchers engaging in the software development life cycle, from product conceptualization to feature identification and user story development to code generation. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04049500; https://clinicaltrials.gov/ct2/show/NCT04049500.

The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians' after-hours clinical work ("work-outside-work"). The surge in patients' digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians' after-hours commitments. We examined the impact on physicians' workload from two types of digital demands - patients' messages requesting medical advice (PMARs) sent to physicians' inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect of PMARs on physicians' work-outside-work and that this relationship is moderated by physicians' specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewer PMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increased PMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. Reducing PMAR volumes and efficient inbasket management strategies needed to reduce physicians' work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.

BACKGROUND:Currently, only about half of U.S. adults achieve current physical activity guidelines. Routine physical activity is not regularly assessed, nor are patients routinely counseled by their health care provider on achieving recommended levels. The three-question physical activity vital sign (PAVS) was developed to assess physical activity duration and intensity and identify adults not meeting physical activity guidelines. Clinical decision support provided via a best practice advisory in an electronic health record (EHR) system can be triggered as a prompt, reminding health care providers to implement the best practice intervention when appropriate. Remote patient monitoring of physical activity can provide objective data in the EHR. OBJECTIVES/OBJECTIVE:This study aimed to evaluate the feasibility and clinical utility of embedding the PAVS and a triggered best practice advisor into the EHR in an ambulatory preventive cardiology practice setting to alert providers to patients reporting low physical activity and prompt health care providers to counsel these patients as needed. METHODS:Three components based in the EHR were integrated for the purpose of this study: patients completed the PAVS through their electronic patient portal prior to an office visit; a best practice advisory was created to prompt providers to counsel patients who reported low levels of physical activity; and remote patient monitoring via Fitbit synced to the EHR provided objective physical activity data. The intervention was pilot-tested in the Epic EHR for 1 year (July 1, 2021-June 30, 2022). Qualitative feedback on the intervention from both providers and patients was obtained at the completion of the study. RESULTS:Monthly assessments of the use of the PAVS and best practice advisory and remote patient monitoring were completed. Patients' completion of the PAVS varied from 35% to 48% per month. The best practice advisory was signed by providers between 2% and 65% and was acknowledged by 2% to 22% per month. The majority (58%) of patients were able to sync a Fitbit device to their EHR for remote monitoring. DISCUSSION/CONCLUSIONS:Although uptake of each component needs improvement, this pilot demonstrated the feasibility of incorporating a PA promotion intervention into the EHR. Qualitative feedback provided guidance for future implementation.

BACKGROUND:Mobile health (mHealth) tools are used to collect data on patient-reported outcomes (PROs) and facilitate the assessment of patients' self-management behaviors outside the clinic environment. Despite the high availability of mHealth diabetes tools, there is a lack of understanding regarding the underlying reasons why these mHealth PRO tools succeed or fail in terms of changing patients' self-management behaviors. OBJECTIVE:This study aims to identify the factors that drive engagement with an mHealth PRO tool and facilitate patients' adoption of self-management behaviors, as well as elicit suggestions for improvement. METHODS:) levels and adherence to self-management behaviors at 12 months among patients with uncontrolled type 2 diabetes. Patients randomized to i-Matter participated in semistructured interviews about their experiences at the 3-, 6-, 9-, and 12-month study visits. A qualitative analysis of the interviews was conducted by 2 experienced qualitative researchers using conventional qualitative content analysis. RESULTS:The sample comprised 71 patients, of whom 67 (94%) completed at least one interview (n=48, 72% female patients; n=25, 37% identified as African American or Black; mean age 56.65 [SD 9.79] years). We identified 4 overarching themes and 6 subthemes. Theme 1 showed that the patients' reasons for engagement with i-Matter were multifactorial. Patients were driven by internal motivating factors that bolstered their engagement and helped them feel accountable for their diabetes (subtheme 1) and external motivating factors that helped to serve as reminders to be consistent with their self-management behaviors (subtheme 2). Theme 2 revealed that the use of i-Matter changed patients' attitudes toward their disease and their health behaviors in 2 ways: patients developed more positive attitudes about their condition and their ability to effectively self-manage it (subtheme 3), and they also developed a better awareness of their current behaviors, which motivated them to adopt healthier lifestyle behaviors (subtheme 4). Theme 3 showed that patients felt more committed to their health as a result of using i-Matter. Theme 4 highlighted the limitations of i-Matter, which included its technical design (subtheme 5) and the need for more resources to support the PRO data collected and shared through the tool (subtheme 6). CONCLUSIONS:This study isolated internal and external factors that prompted patients to change their views about their diabetes, become more engaged with the intervention and their health, and adopt healthy behaviors. These behavioral mechanisms provide important insights to drive future development of mHealth interventions that could lead to sustained behavior change.

BACKGROUND:Remote patient monitoring (RPM) technologies can support patients living with chronic conditions through self-monitoring of physiological measures and enhance clinicians' diagnostic and treatment decisions. However, to date, large-scale pragmatic RPM implementation within health systems has been limited, and understanding of the impacts of RPM technologies on clinical workflows and care experience is lacking. OBJECTIVE:In this study, we evaluate the early implementation of operational RPM initiatives for chronic disease management within the ambulatory network of an academic medical center in New York City, focusing on the experiences of "early adopter" clinicians and patients. METHODS:Using a multimethod qualitative approach, we conducted (1) interviews with 13 clinicians across 9 specialties considered as early adopters and supporters of RPM and (2) speculative design sessions exploring the future of RPM in clinical care with 21 patients and patient representatives, to better understand experiences, preferences, and expectations of pragmatic RPM use for health care delivery. RESULTS:We identified themes relevant to RPM implementation within the following areas: (1) data collection and practices, including impacts of taking real-world measures and issues of data sharing, security, and privacy; (2) proactive and preventive care, including proactive and preventive monitoring, and proactive interventions and support; and (3) health disparities and equity, including tailored and flexible care and implicit bias. We also identified evidence for mitigation and support to address challenges in each of these areas. CONCLUSIONS:This study highlights the unique contexts, perceptions, and challenges regarding the deployment of RPM in clinical practice, including its potential implications for clinical workflows and work experiences. Based on these findings, we offer implementation and design recommendations for health systems interested in deploying RPM-enabled health care.

BACKGROUND:Overprescribing of antibiotics for acute respiratory infections (ARIs) remains a major issue in outpatient settings. Use of clinical prediction rules (CPRs) can reduce inappropriate antibiotic prescribing but they remain underutilized by physicians and advanced practice providers. A registered nurse (RN)-led model of an electronic health record-integrated CPR (iCPR) for low-acuity ARIs may be an effective alternative to address the barriers to a physician-driven model. METHODS:Following qualitative usability testing, we will conduct a stepped-wedge practice-level cluster randomized controlled trial (RCT) examining the effect of iCPR-guided RN care for low acuity patients with ARI. The primary hypothesis to be tested is: Implementation of RN-led iCPR tools will reduce antibiotic prescribing across diverse primary care settings. Specifically, this study aims to: (1) determine the impact of iCPRs on rapid strep test and chest x-ray ordering and antibiotic prescribing rates when used by RNs; (2) examine resource use patterns and cost-effectiveness of RN visits across diverse clinical settings; (3) determine the impact of iCPR-guided care on patient satisfaction; and (4) ascertain the effect of the intervention on RN and physician burnout. DISCUSSION:This study represents an innovative approach to using an iCPR model led by RNs and specifically designed to address inappropriate antibiotic prescribing. This study has the potential to provide guidance on the effectiveness of delegating care of low-acuity patients with ARIs to RNs to increase use of iCPRs and reduce antibiotic overprescribing for ARIs in outpatient settings. TRIAL REGISTRATION:ClinicalTrials.gov Identifier: NCT04255303, Registered February 5 2020, https://clinicaltrials.gov/ct2/show/NCT04255303 .