Development of a computer-aided text message platform for user engagement with a digital Diabetes Prevention Program: a case study
Digital Diabetes Prevention Programs (dDPP) are novel mHealth applications that leverage digital features such as tracking and messaging to support behavior change for diabetes prevention. Despite their clinical effectiveness, long-term engagement to these programs remains a challenge, creating barriers to adherence and meaningful health outcomes. We partnered with a dDPP vendor to develop a personalized automatic message system (PAMS) to promote user engagement to the dDPP platform by sending messages on behalf of their primary care provider. PAMS innovates by integrating into clinical workflows. User-centered design (UCD) methodologies in the form of iterative cycles of focus groups, user interviews, design workshops, and other core UCD activities were utilized to defined PAMS requirements. PAMS uses computational tools to deliver theory-based, automated, tailored messages, and content to support patient use of dDPP. In this article, we discuss the design and development of our system, including key requirements and features, the technical architecture and build, and preliminary user testing.
Patient Interactions With an Automated Conversational Agent Delivering Pretest Genetics Education: Descriptive Study
BACKGROUND:Cancer genetic testing to assess an individual's cancer risk and to enable genomics-informed cancer treatment has grown exponentially in the past decade. Because of this continued growth and a shortage of health care workers, there is a need for automated strategies that provide high-quality genetics services to patients to reduce the clinical demand for genetics providers. Conversational agents have shown promise in managing mental health, pain, and other chronic conditions and are increasingly being used in cancer genetic services. However, research on how patients interact with these agents to satisfy their information needs is limited. OBJECTIVE:Our primary aim is to assess user interactions with a conversational agent for pretest genetics education. METHODS:We conducted a feasibility study of user interactions with a conversational agent who delivers pretest genetics education to primary care patients without cancer who are eligible for cancer genetic evaluation. The conversational agent provided scripted content similar to that delivered in a pretest genetic counseling visit for cancer genetic testing. Outside of a core set of information delivered to all patients, users were able to navigate within the chat to request additional content in their areas of interest. An artificial intelligence-based preprogrammed library was also established to allow users to ask open-ended questions to the conversational agent. Transcripts of the interactions were recorded. Here, we describe the information selected, time spent to complete the chat, and use of the open-ended question feature. Descriptive statistics were used for quantitative measures, and thematic analyses were used for qualitative responses. RESULTS:We invited 103 patients to participate, of which 88.3% (91/103) were offered access to the conversational agent, 39% (36/91) started the chat, and 32% (30/91) completed the chat. Most users who completed the chat indicated that they wanted to continue with genetic testing (21/30, 70%), few were unsure (9/30, 30%), and no patient declined to move forward with testing. Those who decided to test spent an average of 10 (SD 2.57) minutes on the chat, selected an average of 1.87 (SD 1.2) additional pieces of information, and generally did not ask open-ended questions. Those who were unsure spent 4 more minutes on average (mean 14.1, SD 7.41; P=.03) on the chat, selected an average of 3.67 (SD 2.9) additional pieces of information, and asked at least one open-ended question. CONCLUSIONS:The pretest chat provided enough information for most patients to decide on cancer genetic testing, as indicated by the small number of open-ended questions. A subset of participants were still unsure about receiving genetic testing and may require additional education or interpersonal support before making a testing decision. Conversational agents have the potential to become a scalable alternative for pretest genetics education, reducing the clinical demand on genetics providers.
Preferences and patterns of response to public health advice during the COVID-19 pandemic
With recurring waves of the Covid-19 pandemic, a dilemma facing public health leadership is whether to provide public advice that is medically optimal (e.g., most protective against infection if followed), but unlikely to be adhered to, or advice that is less protective but is more likely to be followed. To provide insight about this dilemma, we examined and quantified public perceptions about the tradeoff between (a) the stand-alone value of health behavior advice, and (b) the advice's adherence likelihood. In a series of studies about preference for public health leadership advice, we asked 1061 participants to choose between (5) strict advice that is medically optimal if adhered to but which is less likely to be broadly followed, and (2) relaxed advice, which is less medically effective but more likely to gain adherence-given varying infection expectancies. Participants' preference was consistent with risk aversion. Offering an informed choice alternative that shifts volition to advice recipients only strengthened risk aversion, but also demonstrated that informed choice was preferred as much or more than the risk-averse strict advice.
A Behavioral Economics-Electronic Health Record Module to Promote Appropriate Diabetes Management in Older Adults: Protocol for a Pragmatic Cluster Randomized Controlled Trial
BACKGROUND:The integration of behavioral economics (BE) principles and electronic health records (EHRs) using clinical decision support (CDS) tools is a novel approach to improving health outcomes. Meanwhile, the American Geriatrics Society has created the Choosing Wisely (CW) initiative to promote less aggressive glycemic targets and reduction in pharmacologic therapy in older adults with type 2 diabetes mellitus. To date, few studies have shown the effectiveness of combined BE and EHR approaches for managing chronic conditions, and none have addressed guideline-driven deprescribing specifically in type 2 diabetes. We previously conducted a pilot study aimed at promoting appropriate CW guideline adherence using BE nudges and EHRs embedded within CDS tools at 5 clinics within the New York University Langone Health (NYULH) system. The BE-EHR module intervention was tested for usability, adoption, and early effectiveness. Preliminary results suggested a modest improvement of 5.1% in CW compliance. OBJECTIVE:This paper presents the protocol for a study that will investigate the effectiveness of a BE-EHR module intervention that leverages BE nudges with EHR technology and CDS tools to reduce overtreatment of type 2 diabetes in adults aged 76 years and older, per the CW guideline. METHODS:A pragmatic, investigator-blind, cluster randomized controlled trial was designed to evaluate the BE-EHR module. A total of 66 NYULH clinics will be randomized 1:1 to receive for 18 months either (1) a 6-component BE-EHR module intervention + standard care within the NYULH EHR, or (2) standard care only. The intervention will be administered to clinicians during any patient encounter (eg, in person, telemedicine, medication refill, etc). The primary outcome will be patient-level CW compliance. Secondary outcomes will measure the frequency of intervention component firings within the NYULH EHR, and provider utilization and interaction with the BE-EHR module components. RESULTS:Study recruitment commenced on December 7, 2020, with the activation of all 6 BE-EHR components in the NYULH EHR. CONCLUSIONS:This study will test the effectiveness of a previously developed, iteratively refined, user-tested, and pilot-tested BE-EHR module aimed at providing appropriate diabetes care to elderly adults, compared to usual care via a cluster randomized controlled trial. This innovative research will be the first pragmatic randomized controlled trial to use BE principles embedded within the EHR and delivered using CDS tools to specifically promote CW guideline adherence in type 2 diabetes. The study will also collect valuable information on clinician workflow and interaction with the BE-EHR module, guiding future research in optimizing the timely delivery of BE nudges within CDS tools. This work will address the effectiveness of BE-inspired interventions in diabetes and chronic disease management. TRIAL REGISTRATION/BACKGROUND:ClinicalTrials.gov NCT04181307; https://clinicaltrials.gov/ct2/show/NCT04181307. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)/UNASSIGNED:DERR1-10.2196/28723.
Validation of EHR medication fill data obtained through electronic linkage with pharmacies
Implementing the physical activity vital sign in an academic preventive cardiology clinic
The aims were to implement physical activity (PA) screening as part of the electronic kiosk check-in process in an adult preventive cardiology clinic and assess factors related to patients' self-reported PA. The 3-question physical activity vital sign (PAVS) was embedded in the Epic electronic medical record and included how many days, minutes and intensity (light, moderate, vigorous) of PA patients conducted on average. This is a data analysis of PAVS data over a 60-day period. We conducted multivariable logistic regression to identify factors associated with not meeting current PA recommendations. Over 60Â days, a total of 1322 patients checked into the clinic using the kiosk and 72% (nÂ =Â 951) completed the PAVS at the kiosk. The majority of those patients were male (58%) and White (71%) with a mean age of 64Â Â±Â 15Â years. Of the 951 patients completing the PAVS, 10% reported no PA, 55% reported some PA, and 35% reported achieving at least 150Â min moderate or 75Â min vigorous PA/week. In the logistic model, females (AORÂ =Â 1.4, 95%CI: 1.002-1.8, pÂ =Â .049) vs. males, being Black (AORÂ =Â 2.0, 95%CI: 1.04-3.7, pÂ =Â .038) or 'Other' race (AORÂ =Â 1.5, 95%CI: 1.02-2.3, pÂ =Â .035) vs. White, unknown or other types of relationships (AORÂ =Â 0.0.26, 95%CI: 0.10-0.68, pÂ =Â .006) vs. being married/partnered, and those who were retired (AORÂ =Â 1.9, 95% CI: 1.4-2.8, pÂ <Â .001) or unemployed (AORÂ =Â 2.2, 95%CI: 1.3-3.7, pÂ =Â .002) vs. full-time workers were associated with not achieving recommended levels of PA. The PAVS is a feasible electronic tool for quickly assessing PA and may prompt providers to counsel on this CVD risk factor.
Application of telemedicine video visits in a maternal-fetal medicine practice at the epicenter of the COVID-19 pandemic
BACKGROUND:Telemedicine in obstetrics has mostly been described in the rural areas that have limited access to subspecialties. During the COVID-19 pandemic, health systems rapidly expanded telemedicine services for urgent and nonurgent healthcare delivery, even in urban settings. The New York University health system implemented a prompt systemwide expansion of video-enabled telemedicine visits, increasing telemedicine to >8000 visits daily within 6 weeks of the beginning of the pandemic. There are limited studies that explore patient and provider satisfaction of telemedicine visits in obstetrical patients during the COVID-19 epidemic, particularly in the United States. OBJECTIVE:This study aimed to evaluate both the patients' and the providers' satisfaction with the administration of maternal-fetal medicine services through telemedicine and to identify the factors that drive the patients' desire for future obstetrical telemedicine services. STUDY DESIGN/METHODS:A cross-sectional survey was administered to patients who completed a telemedicine video visit with the Division of Maternal-Fetal Medicine at the New York University Langone Hospital-Long Island from March 19, 2020, to May 26, 2020. A 10-question survey assessing the patients' digital experience and desire for future use was either administered by telephone or self-administered by the patients via a link after obtaining verbal consent. The survey responses were scored from 1-strongly disagree to 5-strongly agree. We analyzed the demographics and survey responses of the patients who agreed to vs those who answered neutral or disagree to the question "I would like telehealth to be an option for future obstetric visits." The providers also answered a similar 10-question survey. The median scores were compared using appropriate tests. A P value of <.05 was considered significant. RESULTS:A total of 253 patients participated in 433 telemedicine visits, and 165 patients completed the survey, resulting in a 65% survey response rate. Overall, there were high rates of patient satisfaction in all areas assessed. Those who desired future telemedicine had significantly greater agreeability that they were able to see and hear their provider easily (5 [4.5, 5] vs 5 [4, 5]; P=.014) and that the lack of physical activity was not an issue (5 [4, 5] vs 5 [4, 5]; P=.032). They were also more likely to agree that the telemedicine visits were as good as in-person visits (4 [3, 5] vs 3 [2, 3]; P<.001) and that telehealth made it easier for them to see doctors or specialists (5 [4, 5] vs 3 [2, 3]; P<.001). The patients seeking consults for poor obstetrical history were more likely to desire future telemedicine compared with other visit types (19 (90%) vs 2 (10%); P=.05). Provider survey responses also demonstrated high levels of satisfaction, with 83% agreeing that they would like telemedicine to be an option for future obstetrical visits. CONCLUSION/CONCLUSIONS:We demonstrated that maternal-fetal medicine obstetrical patients and providers were highly satisfied with the implementation of telemedicine during the initial wave of the COVID-19 pandemic and a majority of them desire telemedicine as an option for future visits. A patient's desire for future telemedicine visits was significantly affected by their digital experience, the perception of a lack of need for physical contact, perceived time saved on travel, and access to healthcare providers. Health systems need to continue to improve healthcare delivery and invest in innovative solutions to conduct physical examinations remotely.
Comparing models of delivery for cancer genetics services among patients receiving primary care who meet criteria for genetic evaluation in two healthcare systems: BRIDGE randomized controlled trial
BACKGROUND:Advances in genetics and sequencing technologies are enabling the identification of more individuals with inherited cancer susceptibility who could benefit from tailored screening and prevention recommendations. While cancer family history information is used in primary care settings to identify unaffected patients who could benefit from a cancer genetics evaluation, this information is underutilized. System-level population health management strategies are needed to assist health care systems in identifying patients who may benefit from genetic services. In addition, because of the limited number of trained genetics specialists and increasing patient volume, the development of innovative and sustainable approaches to delivering cancer genetic services is essential. METHODS:We are conducting a randomized controlled trial, entitled Broadening the Reach, Impact, and Delivery of Genetic Services (BRIDGE), to address these needs. The trial is comparing uptake of genetic counseling, uptake of genetic testing, and patient adherence to management recommendations for automated, patient-directed versus enhanced standard of care cancer genetics services delivery models. An algorithm-based system that utilizes structured cancer family history data available in the electronic health record (EHR) is used to identify unaffected patients who receive primary care at the study sites and meet current guidelines for cancer genetic testing. We are enrolling eligible patients at two healthcare systems (University of Utah Health and New York University Langone Health) through outreach to a randomly selected sample of 2780 eligible patients in the two sites, with 1:1 randomization to the genetic services delivery arms within sites. Study outcomes are assessed through genetics clinic records, EHR, and two follow-up questionnaires at 4â€‰weeks and 12â€‰months after last genetic counseling contactpre-test genetic counseling. DISCUSSION/CONCLUSIONS:BRIDGE is being conducted in two healthcare systems with different clinical structures and patient populations. Innovative aspects of the trial include a randomized comparison of a chatbot-based genetic services delivery model to standard of care, as well as identification of at-risk individuals through a sustainable EHR-based system. The findings from the BRIDGE trial will advance the state of the science in identification of unaffected patients with inherited cancer susceptibility and delivery of genetic services to those patients. TRIAL REGISTRATION/BACKGROUND:BRIDGE is registered as NCT03985852 . The trial was registered on June 6, 2019 at clinicaltrials.gov .
Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials
BACKGROUND:Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools. OBJECTIVE:This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care. METHODS:A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior. RESULTS:To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images. CONCLUSIONS:These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS. TRIAL REGISTRATION/BACKGROUND:Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.
The transformation of patient-clinician relationships with AI-based medical advice
The transformation of patient-clinician relationships with AI-based medical advice is discussed. many new tools are based on entirely new "˜black-box"™ AI-based technologies, whose inner workings are likely not fully understood by patients or clinicians. Most patients with Type 1 diabetes now use continuous glucose monitors and insulin pumps to tightly manage their disease. Their clinicians carefully review the data streams from both devices to recommend dosage adjustments. Recently new automated recommender systems to monitor and analyze food intake, insulin doses, physical activity, and other factors influencing glucose levels, and provide data-intensive, AI-based recommendations on how to titrate the regimen, are in different stages of FDA approval using "˜black box"™ technology, which is an alluring proposition for a clinical scenario that requires identification of meaningful patterns in complex and voluminous data.