Faculty Bibliography

Prostate-specific antigen (PSA)-based prostate cancer screening is a preference-sensitive decision for which experts recommend a shared decision making (SDM) approach. This study aimed to examine PSA screening SDM in primary care. Methods included qualitative analysis of audio-recorded patient-provider interactions supplemented by quantitative description. Participants included 5 clinic providers and 13 patients who were: (1) 40-69 years old, (2) Black, (3) male, and (4) attending clinic for routine primary care. Main measures were SDM element themes and "observing patient involvement in decision making" (OPTION) scoring. Some discussions addressed advantages, disadvantages, and/or scientific uncertainty of screening, however, few patients received all SDM elements. Nearly all providers recommended screening, however, only 3 patients were directly asked about screening preferences. Few patients were asked about prostate cancer knowledge (2), urological symptoms (3), or family history (6). Most providers discussed disadvantages (80%) and advantages (80%) of PSA screening. Average OPTION score was 25/100 (range 0-67) per provider. Our study found limited SDM during PSA screening consultations. The counseling that did take place utilized components of SDM but inconsistently and incompletely. We must improve SDM for PSA screening for diverse patient populations to promote health equity. This study highlights the need to improve SDM for PSA screening.

INTRODUCTION/UNASSIGNED:Shared decision-making (SDM) is a method of care by which patients and clinicians work together to co-create a plan of care. Electronic health record (EHR) integration of SDM tools may increase adoption of SDM. We conducted a "lightweight" integration of a freely available electronic SDM tool, CV Prevention Choice, within the EHRs of three healthcare systems. Here, we report how the healthcare systems collaborated to achieve integration. METHODS/UNASSIGNED:This work was conducted as part of a stepped wedge randomized pragmatic trial. CV Prevention Choice was developed using guidelines for HTML5-based web applications. Healthcare systems integrated the tool in their EHR using documentation the study team developed and refined with lessons learned after each system integrated the electronic SDM tool into their EHR. CV Prevention Choice integration populates the tool with individual patient data locally without sending protected health information between the EHR and the web. Data abstraction and secure transfer systems were developed to manage data collection to assess tool implementation and effectiveness outcomes. RESULTS/UNASSIGNED:Time to integrate CV Prevention Choice in the EHR was 12.1 weeks for the first system, 10.4 weeks for the second, and 9.7 weeks for the third. One system required two 1-hour meetings with study team members and two healthcare systems required a single 1-hour meeting. Healthcare system information technology teams collaborated by sharing information and offering improvements to documentation. Challenges included tracking CV Prevention Choice use for reporting and capture of combination medications. Data abstraction required refinements to address differences in how each healthcare system captured data elements. CONCLUSION/UNASSIGNED:Targeted documentation on tool features and resource mapping supported collaboration of IT teams across healthcare systems, enabling them to integrate a web-based SDM tool with little additional research team effort or oversight. Their collaboration helped overcome difficulties integrating the web application and address challenges to data harmonization for trial outcome analyses.

BACKGROUND:Up to 50% of antibiotic prescriptions for upper respiratory infections (URIs) are inappropriate. Clinical decision support (CDS) systems to mitigate unnecessary antibiotic prescriptions have been implemented into electronic health records, but their use by providers has been limited. OBJECTIVE:As a delegation protocol, we adapted a validated electronic health record-integrated clinical prediction rule (iCPR) CDS-based intervention for registered nurses (RNs), consisting of triage to identify patients with low-acuity URI followed by CDS-guided RN visits. It was implemented in February 2022 as a randomized controlled stepped-wedge trial in 43 primary and urgent care practices within 4 academic health systems in New York, Wisconsin, and Utah. While issues were pragmatically addressed as they arose, a systematic assessment of the barriers to implementation is needed to better understand and address these barriers. METHODS:We performed a retrospective case study, collecting quantitative and qualitative data regarding clinical workflows and triage-template use from expert interviews, study surveys, routine check-ins with practice personnel, and chart reviews over the first year of implementation of the iCPR intervention. Guided by the updated CFIR (Consolidated Framework for Implementation Research), we characterized the initial barriers to implementing a URI iCPR intervention for RNs in ambulatory care. CFIR constructs were coded as missing, neutral, weak, or strong implementation factors. RESULTS:Barriers were identified within all implementation domains. The strongest barriers were found in the outer setting, with those factors trickling down to impact the inner setting. Local conditions driven by COVID-19 served as one of the strongest barriers, impacting attitudes among practice staff and ultimately contributing to a work infrastructure characterized by staff changes, RN shortages and turnover, and competing responsibilities. Policies and laws regarding scope of practice of RNs varied by state and institutional application of those laws, with some allowing more clinical autonomy for RNs. This necessitated different study procedures at each study site to meet practice requirements, increasing innovation complexity. Similarly, institutional policies led to varying levels of compatibility with existing triage, rooming, and documentation workflows. These workflow conflicts were compounded by limited available resources, as well as an implementation climate of optional participation, few participation incentives, and thus low relative priority compared to other clinical duties. CONCLUSIONS:Both between and within health care systems, significant variability existed in workflows for patient intake and triage. Even in a relatively straightforward clinical workflow, workflow and cultural differences appreciably impacted intervention adoption. Takeaways from this study can be applied to other RN delegation protocol implementations of new and innovative CDS tools within existing workflows to support integration and improve uptake. When implementing a system-wide clinical care intervention, considerations must be made for variability in culture and workflows at the state, health system, practice, and individual levels. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04255303; https://clinicaltrials.gov/ct2/show/NCT04255303.

Digital health implementations and investments continue to expand. As the reliance on digital health increases, it is imperative to implement technologies with inclusive and accessible approaches. A conceptual model can be used to guide equity-focused digital health implementations to improve suitability and uptake in diverse populations. The objective of this study is expand an implementation model with recommendations on the equitable implementation of new digital health technologies. The Digital Health Equity-Focused Implementation Research (DH-EquIR) conceptual model was developed based on a rigorous review of digital health implementation and health equity literature. The Equity-Focused Implementation Research for Health Programs (EquIR) model was used as a starting point and merged with digital equity and digital health implementation models. Existing theoretical frameworks and models were appraised as well as individual equity-sensitive implementation studies. Patient and program-related concepts related to digital equity, digital health implementation, and assessment of social/digital determinants of health were included. Sixty-two articles were analyzed to inform the adaption of the EquIR model for digital health. These articles included digital health equity models and frameworks, digital health implementation models and frameworks, research articles, guidelines, and concept analyses. Concepts were organized into EquIR conceptual groupings, including population health status, planning the program, designing the program, implementing the program, and equity-focused implementation outcomes. The adapted DH-EquIR conceptual model diagram was created as well as detailed tables displaying related equity concepts, evidence gaps in source articles, and analysis of existing equity-related models and tools. The DH-EquIR model serves to guide digital health developers and implementation specialists to promote the inclusion of health-equity planning in every phase of implementation. In addition, it can assist researchers and product developers to avoid repeating the mistakes that have led to inequities in the implementation of digital health across populations.

BACKGROUND:Currently, only about half of U.S. adults achieve current physical activity guidelines. Routine physical activity is not regularly assessed, nor are patients routinely counseled by their health care provider on achieving recommended levels. The three-question physical activity vital sign (PAVS) was developed to assess physical activity duration and intensity and identify adults not meeting physical activity guidelines. Clinical decision support provided via a best practice advisory in an electronic health record (EHR) system can be triggered as a prompt, reminding health care providers to implement the best practice intervention when appropriate. Remote patient monitoring of physical activity can provide objective data in the EHR. OBJECTIVES/OBJECTIVE:This study aimed to evaluate the feasibility and clinical utility of embedding the PAVS and a triggered best practice advisor into the EHR in an ambulatory preventive cardiology practice setting to alert providers to patients reporting low physical activity and prompt health care providers to counsel these patients as needed. METHODS:Three components based in the EHR were integrated for the purpose of this study: patients completed the PAVS through their electronic patient portal prior to an office visit; a best practice advisory was created to prompt providers to counsel patients who reported low levels of physical activity; and remote patient monitoring via Fitbit synced to the EHR provided objective physical activity data. The intervention was pilot-tested in the Epic EHR for 1 year (July 1, 2021-June 30, 2022). Qualitative feedback on the intervention from both providers and patients was obtained at the completion of the study. RESULTS:Monthly assessments of the use of the PAVS and best practice advisory and remote patient monitoring were completed. Patients' completion of the PAVS varied from 35% to 48% per month. The best practice advisory was signed by providers between 2% and 65% and was acknowledged by 2% to 22% per month. The majority (58%) of patients were able to sync a Fitbit device to their EHR for remote monitoring. DISCUSSION/CONCLUSIONS:Although uptake of each component needs improvement, this pilot demonstrated the feasibility of incorporating a PA promotion intervention into the EHR. Qualitative feedback provided guidance for future implementation.

OBJECTIVES/OBJECTIVE:To evaluate demographic biases in diagnostic accuracy and health advice between generative artificial intelligence (AI) (ChatGPT GPT-4) and traditional symptom checkers like WebMD. MATERIALS AND METHODS/METHODS:Combination symptom and demographic vignettes were developed for 27 most common symptom complaints. Standardized prompts, written from a patient perspective, with varying demographic permutations of age, sex, and race/ethnicity were entered into ChatGPT (GPT-4) between July and August 2023. In total, 3 runs of 540 ChatGPT prompts were compared to the corresponding WebMD Symptom Checker output using a mixed-methods approach. In addition to diagnostic correctness, the associated text generated by ChatGPT was analyzed for readability (using Flesch-Kincaid Grade Level) and qualitative aspects like disclaimers and demographic tailoring. RESULTS:ChatGPT matched WebMD in 91% of diagnoses, with a 24% top diagnosis match rate. Diagnostic accuracy was not significantly different across demographic groups, including age, race/ethnicity, and sex. ChatGPT's urgent care recommendations and demographic tailoring were presented significantly more to 75-year-olds versus 25-year-olds (P < .01) but were not statistically different among race/ethnicity and sex groups. The GPT text was suitable for college students, with no significant demographic variability. DISCUSSION/CONCLUSIONS:The use of non-health-tailored generative AI, like ChatGPT, for simple symptom-checking functions provides comparable diagnostic accuracy to commercially available symptom checkers and does not demonstrate significant demographic bias in this setting. The text accompanying differential diagnoses, however, suggests demographic tailoring that could potentially introduce bias. CONCLUSION/CONCLUSIONS:These results highlight the need for continued rigorous evaluation of AI-driven medical platforms, focusing on demographic biases to ensure equitable care.

BACKGROUND:Generative artificial intelligence has the potential to revolutionize health technology product development by improving coding quality, efficiency, documentation, quality assessment and review, and troubleshooting. OBJECTIVE:This paper explores the application of a commercially available generative artificial intelligence tool (ChatGPT) to the development of a digital health behavior change intervention designed to support patient engagement in a commercial digital diabetes prevention program. METHODS:We examined the capacity, advantages, and limitations of ChatGPT to support digital product idea conceptualization, intervention content development, and the software engineering process, including software requirement generation, software design, and code production. In total, 11 evaluators, each with at least 10 years of experience in fields of study ranging from medicine and implementation science to computer science, participated in the output review process (ChatGPT vs human-generated output). All had familiarity or prior exposure to the original personalized automatic messaging system intervention. The evaluators rated the ChatGPT-produced outputs in terms of understandability, usability, novelty, relevance, completeness, and efficiency. RESULTS:Most metrics received positive scores. We identified that ChatGPT can (1) support developers to achieve high-quality products faster and (2) facilitate nontechnical communication and system understanding between technical and nontechnical team members around the development goal of rapid and easy-to-build computational solutions for medical technologies. CONCLUSIONS:ChatGPT can serve as a usable facilitator for researchers engaging in the software development life cycle, from product conceptualization to feature identification and user story development to code generation. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04049500; https://clinicaltrials.gov/ct2/show/NCT04049500.

BACKGROUND:Digital diabetes prevention programs (dDPPs) are effective "digital prescriptions" but have high attrition rates and program noncompletion. To address this, we developed a personalized automatic messaging system (PAMS) that leverages SMS text messaging and data integration into clinical workflows to increase dDPP engagement via enhanced patient-provider communication. Preliminary data showed positive results. However, further investigation is needed to determine how to optimize the tailoring of support technology such as PAMS based on a user's preferences to boost their dDPP engagement. OBJECTIVE:This study evaluates leveraging machine learning (ML) to develop digital engagement phenotypes of dDPP users and assess ML's accuracy in predicting engagement with dDPP activities. This research will be used in a PAMS optimization process to improve PAMS personalization by incorporating engagement prediction and digital phenotyping. This study aims (1) to prove the feasibility of using dDPP user-collected data to build an ML model that predicts engagement and contributes to identifying digital engagement phenotypes, (2) to describe methods for developing ML models with dDPP data sets and present preliminary results, and (3) to present preliminary data on user profiling based on ML model outputs. METHODS:Using the gradient-boosted forest model, we predicted engagement in 4 dDPP individual activities (physical activity, lessons, social activity, and weigh-ins) and general activity (engagement in any activity) based on previous short- and long-term activity in the app. The area under the receiver operating characteristic curve, the area under the precision-recall curve, and the Brier score metrics determined the performance of the model. Shapley values reflected the feature importance of the models and determined what variables informed user profiling through latent profile analysis. RESULTS:We developed 2 models using weekly and daily DPP data sets (328,821 and 704,242 records, respectively), which yielded predictive accuracies above 90%. Although both models were highly accurate, the daily model better fitted our research plan because it predicted daily changes in individual activities, which was crucial for creating the "digital phenotypes." To better understand the variables contributing to the model predictor, we calculated the Shapley values for both models to identify the features with the highest contribution to model fit; engagement with any activity in the dDPP in the last 7 days had the most predictive power. We profiled users with latent profile analysis after 2 weeks of engagement (Bayesian information criterion=-3222.46) with the dDPP and identified 6 profiles of users, including those with high engagement, minimal engagement, and attrition. CONCLUSIONS:Preliminary results demonstrate that applying ML methods with predicting power is an acceptable mechanism to tailor and optimize messaging interventions to support patient engagement and adherence to digital prescriptions. The results enable future optimization of our existing messaging platform and expansion of this methodology to other clinical domains. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04773834; https://www.clinicaltrials.gov/ct2/show/NCT04773834. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)/UNASSIGNED:RR2-10.2196/26750.

The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians' after-hours clinical work ("work-outside-work"). The surge in patients' digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians' after-hours commitments. We examined the impact on physicians' workload from two types of digital demands - patients' messages requesting medical advice (PMARs) sent to physicians' inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect of PMARs on physicians' work-outside-work and that this relationship is moderated by physicians' specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewer PMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increased PMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. Reducing PMAR volumes and efficient inbasket management strategies needed to reduce physicians' work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.