Faculty Bibliography

BACKGROUND:Among the alternative solutions being tested to improve access to genetic services, chatbots (or conversational agents) are being increasingly used for service delivery. Despite the growing number of studies on the accessibility and feasibility of chatbot genetic service delivery, limited attention has been paid to user interactions with chatbots in a real-world health care context. OBJECTIVE:We examined users' interaction patterns with a pretest cancer genetics education chatbot as well as the associations between users' clinical and sociodemographic characteristics, chatbot interaction patterns, and genetic testing decisions. METHODS:We analyzed data from the experimental arm of Broadening the Reach, Impact, and Delivery of Genetic Services, a multisite genetic services pragmatic trial in which participants eligible for hereditary cancer genetic testing based on family history were randomized to receive a chatbot intervention or standard care. In the experimental chatbot arm, participants were offered access to core educational content delivered by the chatbot with the option to select up to 9 supplementary informational prompts and ask open-ended questions. We computed descriptive statistics for the following interaction patterns: prompt selections, open-ended questions, completion status, dropout points, and postchat decisions regarding genetic testing. Logistic regression models were used to examine the relationships between clinical and sociodemographic factors and chatbot interaction variables, examining how these factors affected genetic testing decisions. RESULTS:Of the 468 participants who initiated a chat, 391 (83.5%) completed it, with 315 (80.6%) of the completers expressing a willingness to pursue genetic testing. Of the 391 completers, 336 (85.9%) selected at least one informational prompt, 41 (10.5%) asked open-ended questions, and 3 (0.8%) opted for extra examples of risk information. Of the 77 noncompleters, 57 (74%) dropped out before accessing any informational content. Interaction patterns were not associated with clinical and sociodemographic factors except for prompt selection (varied by study site) and completion status (varied by family cancer history type). Participants who selected ≥3 prompts (odds ratio 0.33, 95% CI 0.12-0.91; P=.03) or asked open-ended questions (odds ratio 0.46, 95% CI 0.22-0.96; P=.04) were less likely to opt for genetic testing. CONCLUSIONS:Findings highlight the chatbot's effectiveness in engaging users and its high acceptability, with most participants completing the chat, opting for additional information, and showing a high willingness to pursue genetic testing. Sociodemographic factors were not associated with interaction patterns, potentially indicating the chatbot's scalability across diverse populations provided they have internet access. Future efforts should address the concerns of users with high information needs and integrate them into chatbot design to better support informed genetic decision-making.

Background Overuse and misuse of antibiotics is an urgent healthcare problem and one of the key factors in antibiotic resistance. Validated clinical prediction rules have shown effectiveness in guiding providers to an appropriate diagnosis and identifying when antibiotics are the recommended choice for treatment. Objective We aimed to study the relative ability of registered nurses using clinical prediction rules to guide the management of acute respiratory infections in a simulated environment compared to practicing primary care physicians. Design We evaluated a case-based simulation of the diagnosis and treatment for acute respiratory infections using clinical prediction rules. As a secondary outcome, we examined nursing self-efficacy by administering a survey before and after case evaluations. Participants Participants included 40 registered nurses from three academic medical centers and five primary care physicians as comparators. Participants evaluated six simulated case studies, three for patients presenting with cough symptoms and three for sore throat. Key Results Compared to physicians, nurses determined risk and treatment for simulated sore throat cases using clinical prediction rules with nurses having 100% accuracy in low-risk sore throat cases versus 80% for physicians. We found great variability in the accuracy of the risk level and appropriate treatment for cough cases. Nurses reported slight increases in self-efficacy from baseline to post-case evaluation suggesting further information is needed to understand correlation. Conclusions Clinical prediction rules used by nurses in sore throat management workflows can guide accurate diagnosis and treatment in simulated cases, while cough management requires further exploration. Our results support the future implementation of automated prediction rules in a clinical decision support tool and a thorough examination of their effect on clinical practice and patient outcomes.

Uncontrolled hypertension is common and frequently related to inadequate adherence to prescribed medications, resulting in suboptimal blood pressure control and increased healthcare utilization. Although healthcare providers have the opportunity to improve medication adherence, they may lack the tools to address adherence at the point of care. This study aims to assess the usability of a digital tool designed to improve medication adherence and blood pressure control among patients with hypertension who are not adherent to therapy. By evaluating usability, the study seeks to refine the tool's design, underscore the role of technology in managing hypertension, and provide insights to inform clinical decisions.We performed qualitative usability testing of an electronic health record (EHR)-integrated intervention with medical assistants (MAs) and primary care providers (PCPs) from a large integrated health system. Usability was assessed with these end-users using the "think aloud" and "near live" approaches. This evaluation was guided by two frameworks: the End-User Computing Satisfaction Index (EUCSI) and the Technology Acceptance Model (TAM). Interviews were analyzed using a thematic analysis approach.Thematic saturation was reached after usability testing was performed with 10 participants, comprising 5 PCPs and 5 MAs. The study identified several strengths within the content, format, ease of use, timeliness, accuracy, and usefulness of the tool, including the user-friendly content presentation, the usefulness of adherence information, and timely alerts that fit into the workflow. Challenges centered around alert visibility and specificity of information.Leveraging the two conceptual frameworks (TAM and EUCSI) to test the usability of the medication adherence tool was helpful. The tool's several strengths and opportunities for improvement were found. The resulting suggestions will be used to support the enhancement of the design for optimal implementation in a clinical trial.

IMPORTANCE/UNASSIGNED:Medication nonadherence is present in nearly half of patients with hypertension but is underrecognized in clinical care. Data linkages between electronic health records and pharmacies have created opportunities for scalable assessment of medication adherence at the point of care. OBJECTIVE/UNASSIGNED:To test the effectiveness of a multicomponent intervention that identified patients with uncontrolled hypertension and medication nonadherence using linked electronic health record-pharmacy data combined with team-based care to address adherence barriers. DESIGN, SETTING, AND PARTICIPANTS/UNASSIGNED:TEAMLET (Leveraging Electronic Health Record Technology and Team Care to Address Medication Adherence) was a pragmatic, 2-arm, cluster randomized clinical trial conducted between October 2022 and November 2024 in 10 primary care sites in New York. The study included adults with uncontrolled hypertension and low medication adherence, defined as proportion of days covered (PDC) less than 80%. Data analysis was performed from November 2024 to January 2025. INTERVENTION/UNASSIGNED:The intervention consisted of the following: (1) automated identification of patients with medication nonadherence at the time of the visit; (2) prompting of medical assistants to screen for barriers to adherence; (3) clinical decision support alerting the primary care physicians and nurse practitioners to barriers to adherence; and (4) adherence discussion between the primary care physician or nurse practitioner and the patient. The comparator was usual care. MAIN OUTCOMES AND MEASURES/UNASSIGNED:The primary outcome was change in PDC from baseline to 12 months. RESULTS/UNASSIGNED:Among 1726 patients (mean [SD] age, 67.2 [13.9] years; 887 [51.4%] female), the mean (SD) baseline PDC was 33.2% (30.5%) overall (32.4% [30.4%] in the intervention group and 34.0% [30.6%] in the control group). The mean (SD) PDC at 12 months was 51.1% (39.5%) for the intervention group and 53.1% (39.6%) for the control group. No difference was found in the change in PDC from baseline to 12 months between the intervention and control groups (mean [SD] absolute change in PDC, 18.5 [41.1] vs 18.2 [40.9] percentage points, respectively; adjusted difference, -0.15 percentage point; 95% CI, -4.06 to 3.76 percentage points). Change in systolic blood pressure and patients who became adherent (PDC ≥80%) at 12 months were also similar between groups. CONCLUSIONS AND RELEVANCE/UNASSIGNED:In this pragmatic trial, an intervention that combined team-based primary care with automated identification of patients with antihypertensive medication nonadherence did not lead to improvements in adherence or blood pressure. TRIAL REGISTRATION/UNASSIGNED:ClinicalTrials.gov Identifier: NCT05349422.

OBJECTIVE:To examine whether patient sociodemographic and clinical characteristics and prior interactions with the healthcare system were associated with opening patient portal messages related to cancer genetic services and beginning services. STUDY SETTING AND DESIGN/METHODS:The trial was conducted in the University of Utah Health (UHealth) and NYU Langone Health (NYULH) systems. Between 2020 and 2023, 3073 eligible primary care patients aged 25-60 years meeting family history-based criteria for cancer genetic evaluation were randomized 1:1 to receive a patient portal message with a hyperlink to a pretest genetics education chatbot or information about scheduling a pretest standard of care (SOC) appointment. DATA SOURCES AND ANALYTIC SAMPLE/UNASSIGNED:Primary data were collected. Eligible patients had a primary care visit in the previous 3 years, a patient portal account, no prior cancer diagnosis except nonmelanoma skin cancer, no prior cancer genetic services, and English or Spanish as their preferred language. Multivariable models identified predictors of opening patient portal messages by site and beginning pretest genetic services by site and experimental condition. PRINCIPAL FINDINGS/RESULTS:Number of previous patient portal logins (UHealth average marginal effect [AME]: 0.32; 95% CI: 0.27, 0.38; NYULH AME: 0.33; 95% CI: 0.27, 0.39), having a recorded primary care provider (NYULH AME: 0.15; 95% CI: 0.08, 0.22), and more primary care visits in the previous 3 years (NYULH AME: 0.09; 95% CI: 0.02, 0.16) were associated with opening patient portal messages about genetic services. Number of previous patient portal logins (UHealth AME: 0.14; 95% CI: 0.08, 0.21; NYULH AME: 0.18; 95% CI: 0.12, 0.23), having a recorded primary care provider (NYULH AME: 0.08; 95% CI: 0.01, 0.14), and more primary care visits in the previous 3 years (NYULH AME: 0.07; 95% CI: 0.01, 0.13) were associated with beginning pretest genetic services. Patient sociodemographic and clinical characteristics were not significantly associated with either outcome. CONCLUSIONS:As system-level initiatives aim to reach patients eligible for cancer genetic services, patients already interacting with the healthcare system may be most likely to respond. Addressing barriers to accessing healthcare and technology may increase engagement with genetic services.

OBJECTIVE:To evaluate trends in telemedicine utilization overall and across clinical specialties, providing insights into its evolving role in health care delivery. STUDY DESIGN/METHODS:This retrospective cross-sectional study analyzed 1.9 million telemedicine video visits from a large academic health care system in New York City between 2020 and 2023. The data, collected from the health care system's electronic health records, included telemedicine encounters across more than 500 ambulatory locations. METHODS:We used descriptive statistics to outline telemedicine usage trends and compared telemedicine utilization rates and evaluation and management characteristics across clinical specialties. RESULTS:Telemedicine utilization peaked during the COVID-19 pandemic, then declined and stabilized. Despite an overall decline, 2 non-primary care specialties (behavioral health and psychiatry) experienced continued growth in telemedicine visits. Primary care and urgent care visits were mainly characterized by low-complexity visits, whereas non-primary care specialties witnessed a rise in moderate- and high-complexity visits, with the number of moderate-level visits surpassing those of low complexity. CONCLUSIONS:The findings highlight a dynamic shift in telemedicine utilization, with non-primary care settings witnessing an increase in the complexity of cases. To address future demands from increasingly complex medical cases managed through telemedicine in non-primary care, appropriate resource allocation is essential.

BACKGROUND/UNASSIGNED:Usability testing is valuable for assessing a new tool or system's usefulness and ease-of-use. Several established methods of usability testing exist, including think-aloud testing. Although usability testing has been shown to be crucial for successful clinical decision support (CDS) tool development, it is often difficult to conduct across multisite development projects due to its time- and labor-intensiveness, cost, and the skills required to conduct the testing. OBJECTIVE/UNASSIGNED:Our objective was to develop a new method of usability testing that would enable efficient acquisition and dissemination of results among multiple sites. We sought to address the existing barriers to successfully completing usability testing during CDS tool development. METHODS/UNASSIGNED:We combined individual think-aloud testing and focus groups into one session and performed sessions serially across 4 sites (snowball group usability testing) to assess the usability of two CDS tools designed for use by nurses in primary and urgent care settings. We recorded each session and took notes in a standardized format. Each site shared feedback from their individual sessions with the other sites in the study so that they could incorporate that feedback into their tools prior to their own testing sessions. RESULTS/UNASSIGNED:The group testing and snowballing components of our new usability testing method proved to be highly beneficial. We identified 3 main benefits of snowball group usability testing. First, by interviewing several participants in a single session rather than individuals over the course of weeks, each site was able to quickly obtain their usability feedback. Second, combining the individualized think-aloud component with a focus group component in the same session helped study teams to more easily notice similarities in feedback among participants and to discuss and act upon suggestions efficiently. Third, conducting usability testing in series across sites allowed study teams to incorporate feedback based on previous sites' sessions prior to conducting their own testing. CONCLUSIONS/UNASSIGNED:Snowball group usability testing provides an efficient method of obtaining multisite feedback on newly developed tools and systems, while addressing barriers typically associated with traditional usability testing methods. This method can be applied to test a wide variety of tools, including CDS tools, prior to launch so that they can be efficiently optimized.

BACKGROUND:Laypeople have easy access to health information through large language models (LLMs), such as ChatGPT, and search engines, such as Google. Search engines transformed health information access, and LLMs offer a new avenue for answering laypeople's questions. OBJECTIVE:We aimed to compare the frequency of use and attitudes toward LLMs and search engines as well as their comparative relevance, usefulness, ease of use, and trustworthiness in responding to health queries. METHODS:We conducted a screening survey to compare the demographics of LLM users and nonusers seeking health information, analyzing results with logistic regression. LLM users from the screening survey were invited to a follow-up survey to report the types of health information they sought. We compared the frequency of use of LLMs and search engines using ANOVA and Tukey post hoc tests. Lastly, paired-sample Wilcoxon tests compared LLMs and search engines on perceived usefulness, ease of use, trustworthiness, feelings, bias, and anthropomorphism. RESULTS:In total, 2002 US participants recruited on Prolific participated in the screening survey about the use of LLMs and search engines. Of them, 52% (n=1045) of the participants were female, with a mean age of 39 (SD 13) years. Participants were 9.7% (n=194) Asian, 12.1% (n=242) Black, 73.3% (n=1467) White, 1.1% (n=22) Hispanic, and 3.8% (n=77) were of other races and ethnicities. Further, 1913 (95.6%) used search engines to look up health queries versus 642 (32.6%) for LLMs. Men had higher odds (odds ratio [OR] 1.63, 95% CI 1.34-1.99; P<.001) of using LLMs for health questions than women. Black (OR 1.90, 95% CI 1.42-2.54; P<.001) and Asian (OR 1.66, 95% CI 1.19-2.30; P<.01) individuals had higher odds than White individuals. Those with excellent perceived health (OR 1.46, 95% CI 1.1-1.93; P=.01) were more likely to use LLMs than those with good health. Higher technical proficiency increased the likelihood of LLM use (OR 1.26, 95% CI 1.14-1.39; P<.001). In a follow-up survey of 281 LLM users for health, most participants used search engines first (n=174, 62%) to answer health questions, but the second most common first source consulted was LLMs (n=39, 14%). LLMs were perceived as less useful (P<.01) and less relevant (P=.07), but elicited fewer negative feelings (P<.001), appeared more human (LLM: n=160, vs search: n=32), and were seen as less biased (P<.001). Trust (P=.56) and ease of use (P=.27) showed no differences. CONCLUSIONS:Search engines are the primary source of health information; yet, positive perceptions of LLMs suggest growing use. Future work could explore whether LLM trust and usefulness are enhanced by supplementing answers with external references and limiting persuasive language to curb overreliance. Collaboration with health organizations can help improve the quality of LLMs' health output.

INTRODUCTION/BACKGROUND:Early lung cancer screening (LCS) through low-dose CT (LDCT) is crucial but underused due to various barriers, including incomplete or inaccurate patient smoking data in the electronic health record and limited time for shared decision-making. The objective of this trial is to investigate a patient-centred intervention, MyLungHealth, delivered through the patient portal. The intervention is designed to improve LCS rates through increased identification of eligible patients and informed decision-making. METHODS AND ANALYSIS/METHODS:MyLungHealth is a multisite pragmatic trial, involving University of Utah Health and New York University Langone Health primary care clinics. The MyLungHealth intervention was developed using a user-centred design process, informed by patient and provider focus groups and interviews. The intervention's effectiveness will be evaluated through a patient-randomised trial, comparing the combined use of MyLungHealth and DecisionPrecision+ (a provider-focused shared decision-making intervention) against DecisionPrecision+ alone. The first study hypothesis is that among patients aged 50-79 with uncertain LCS eligibility (eg, 10-19 pack-years or unknown pack-years or unknown quit date for individuals who used to smoke), MyLungHealth eligibility questionnaires will result in increased identification of LCS-eligible patients (n~26 729 patients). The second study hypothesis is that among patients aged 50-79 with documented LCS eligibility (20+ pack-years, quit within the last 15 years if individuals who used to smoke, and no recent screening or screening discussion), MyLungHealth education will result in increased LDCT ordering (n~4574 patients). Primary outcomes will be identification of LCS-eligible patients among individuals with uncertain LCS eligibility and LDCT ordering rates among individuals with documented LCS eligibility. ETHICS AND DISSEMINATION/BACKGROUND:The protocol was approved by the University of Utah Institutional Review Board (# 00153806). The patient data collected for this study will not be shared publicly due to the sensitive nature of the patient health information and the fact that we will not be obtaining written informed consent to allow public sharing of their data. Results will be disseminated through peer-reviewed publications. TRIAL REGISTRATION NUMBER/BACKGROUND:Clinicaltrials.gov, NCT06338592.