Faculty Bibliography

Ensuring reliability of Large Language Models (LLMs) in clinical tasks is crucial. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR. Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation. For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date. In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

CONTEXT/BACKGROUND:The critical role of hospice and palliative care in response to the COVID-19 pandemic is well recognized, but there is limited evidence to guide healthcare leadership through future crises. OBJECTIVES/OBJECTIVE:Our goal was to support future organizational resilience by exploring hospice and palliative team members' perspectives on crisis leadership during the COVID-19 pandemic in New York City (NYC). METHODS:This qualitative descriptive study used individual, semi-structured interviews of purposively sampled interdisciplinary team members. Enrollment sites were two large NYC metro hospice care organizations and one outpatient palliative care practice. We asked participants to complete a demographic form and a 45-60 minute interview. We used descriptive statistics and thematic analysis, respectively, for data analysis. We triangulated the data by presenting preliminary study findings to a group of clinicians (n=21) from one of the referring organizations. RESULTS:Participants (n=30) were professionally diverse (e.g., nurses, physicians, social workers, chaplains, administrators), experienced (mean=17 years; 10 years in hospice), and highly educated (83% ≥ master's degree). About half (n=15) self-identified as white, non-Hispanic, and nearly half (n=13) self-identified as being from a racial/ethnic minoritized group. Two (n=2) did not wish to self-identify. We identified four themes that reflected challenges and adaptive responses to providing care during a crisis: Stay Open and Stay Safe; Act Flexibly; Lead Adaptively; and Create a Culture of Solidarity. CONCLUSION/CONCLUSIONS:While additional work is indicated, findings offer direction for a crisis management playbook to guide leadership in hospice, palliative care, and other healthcare settings in future crises.

BACKGROUND AND OBJECTIVES/OBJECTIVE:The coronavirus disease 2019 (COVID-19) pandemic severely disrupted hospice care, yet there is little research regarding how widespread disruptions affected clinician and family decision-making. We aimed to understand how the pandemic affected structures, processes, and outcomes of end-of-life care. RESEARCH DESIGN AND METHODS/METHODS:Retrospective narrative chart review of electronic health records of 61 patients referred and admitted to hospice from 3 New York City geriatrics practices who died between March 1, 2020, and March 31, 2021. We linked longitudinal, unstructured medical, and hospice electronic health record notes to create a real-time, multiperspective trajectory of patients' interactions with providers using directed content analysis. RESULTS:Most patients had dementia and were enrolled in hospice for 11 days. Care processes were shaped by structural factors (staffing, supplies, and governmental/institutional policies), and outcomes were prioritized by care teams and families (protecting safety, maintaining high-touch care, honoring patient values, and supporting patients emotionally and spiritually). Processes used to achieve these outcomes were decision-making, care delivery, supporting a "good death," and emotional and spiritual support. DISCUSSION AND IMPLICATIONS/CONCLUSIONS:Care processes were negotiated throughout the end of life, with clinicians and families making in-the-moment decisions. Some adaptations were effective but also placed extraordinary pressure on paid and family caregivers. Healthcare teams' and families' goals to meet patients' end-of-life priorities can be supported by ongoing assessment of patient goals and process changes needed to support them, stronger structural supports for paid and family caregivers, incentivizing relationships across primary care and hospice teams, and extending social work and spiritual care.

BACKGROUND:the perspectives of older adults undergoing the TAVR process have never been synthesized. OBJECTIVES/OBJECTIVE:To contextualize the perspectives and experiences of older adults undergoing TAVR. METHODS:Four electronic databases were searched in April 2023. Articles were included if a qualitative methodology was used to assess the perceptions of older adults (>65 years old) undergoing or recovering from TAVR. RESULTS:Out of 4619 articles screened, 12 articles met the criteria, representing 353 individuals from 10 countries. Relevant themes included the need for an individualized care plan, caregiver and family support, communication and education, persistent psychosocial and physical symptoms, and the unique recovery journey. CONCLUSION/CONCLUSIONS:Older adults with AS undergoing TAVR generally perceive their procedure positively. Improved interdisciplinary and holistic management, open communication, symptom assessment, support, and education is needed.

The purpose of this article was to analyze the concept of "the fighter in the intensive care unit (ICU)" per the scientific literature and the impact this mentality has on care administered in the ICU. A literature review and a concept analysis based on Rodger's evolutionary method were performed to identify surrogate terms, antecedents, attributes, and consequences pertaining to the "fighter" in the ICU. Thirteen articles with a focus on "the fighter" were included in this analysis. There is a strong desire to remain optimistic and maintain high spirits as a coping mechanism in the face of extreme prognostic uncertainty. Themes that emerged from the literature were the need to find inner strength and persist in the face of adversity. The concept of "the fighter in the ICU" can serve as either adaptive or maladaptive coping, depending on the larger clinical picture. Patient experiences in the ICU are fraught with physical and psychological distress. How the patient and family unit cope during this anxiety-provoking time is based on the individual. Maintaining optimism and identifying as a fighter can be healthy ways to adapt to the circumstances. This concept analysis highlights the importance of holistic care and instilling hope particularly as patients may be nearing the end of life.

OBJECTIVES/OBJECTIVE:Medicare Home Health Care (HHC) services are integral to the care of homebound seriously ill older adults requiring ongoing specialized medical care. Although disparities in health outcomes are well documented in inpatient and primary care, disparities experienced by historically marginalized racial and ethnic groups underrepresented in HHC are understudied. This study aimed to examine the relationship between individual characteristics and differences in HHC health outcomes for seriously ill older adults. DESIGN/METHODS:Secondary data analysis, repeated measure. SETTING AND PARTICIPANTS/METHODS:Seriously ill older adults who received HHC in 2016 in the HHC Outcome and Assessment Information Set (OASIS). METHODS:Start of care and discharge data from the 2016 HCC OASIS were used to examine the relationship between individual characteristics and differences in HHC health outcomes identified by the Centers for Medicare and Medicaid Services as key indicators of quality in HHC, including dyspnea, pain frequency, cognitive functioning, and presence of unhealed pressure ulcer stage II or higher. A generalized ordered logit model with partial proportional odds was used for the ordinal categorical outcomes and a logistic regression was used for the binary dependent variable. RESULTS:Findings indicated that of 227,402 seriously ill individuals with an HHC episode in 2016, those from underrepresented racial and ethnic groups had between 14% and 57% higher odds of worse health outcomes compared with non-Hispanic white patients with the exception of pain frequency. CONCLUSIONS AND IMPLICATIONS/CONCLUSIONS:For people living with serious illness, there are significant differences in Medicare HHC health outcomes when comparing underrepresented racial or ethnic beneficiaries with white counterparts. More research is needed to understand how health care processes such as referral patterns or time to care initiation, and structural factors such as HHC agency quality and neighborhood social deprivation are related to health differences observed in the population.

OBJECTIVES/OBJECTIVE:To evaluate the impact of a mentoring program to encourage staff-delivered sleep-promoting strategies on sleep, function, depression, and anxiety among skilled nursing facility (SNF) residents. DESIGN/METHODS:Modified stepped-wedge unit-level intervention. SETTING AND PARTICIPANTS/METHODS:Seventy-two residents (mean age 75 ± 15 years; 55.9% female, 41% non-Hispanic White, 35% Black, 20% Hispanic, 3% Asian) of 2 New York City urban SNFs. METHODS:. RESULTS:. CONCLUSIONS AND IMPLICATIONS/CONCLUSIONS:SNF residents had improvements in sleep quality and depression with intervention, but improvements were not sustained at 3-month follow-up. The COVID-19 pandemic led to premature study termination, so full impacts remain unknown.

OBJECTIVES/OBJECTIVE:To describe the implementation of a mentored staff-delivered sleep program in nursing facilities. DESIGN/METHODS:Modified stepped-wedge unit-level intervention. SETTING AND PARTICIPANTS/METHODS:This program was implemented in 2 New York City nursing facilities, with partial implementation (due to COVID-19) in a third facility. METHODS:Expert mentors provided staff webinars, in-person workshops, and weekly sleep pearls via text messaging. We used the integrated Promoting Action on Research Implementation in Health Services (i-PARiHS) framework as a post hoc approach to describe key elements of the SLUMBER implementation. We measured staff participation in unit-level procedures and noted their commentary during unit workshops. RESULTS:We completed SLUMBER within 5 units across 2 facilities and held 15 leadership meetings before and during program implementation. Sessions on each unit included 3 virtual webinar presentations and 4 in-person workshops for each nursing shift, held over a period of 3 to 4 months. Staff attendance averaged >3 sessions per individual staff member. Approximately 65% of staff present on each unit participated in any given session. Text messaging was useful for engagement, educational reinforcement, and encouraging attendance. We elevated staff as experts in the care of their residents as a strategy for staff engagement and behavior change and solicited challenging cases from staff during workshops to provide strategies to address resident behavior and encourage adoption when successful. CONCLUSIONS AND IMPLICATIONS/CONCLUSIONS:Engaging staff, leadership, residents, and family of nursing facilities in implementing a multicomponent sleep quality improvement program is feasible for improving nursing facilities' sleep environment. The program required gaining trust at multiple levels through presence and empathy, and reinforcement mechanisms (primarily text messages). To improve scalability, SLUMBER could evolve from an interdisciplinary investigator-based approach to internal coaches in a train-the-trainer model to effectively and sustainably implement this program to improve sleep quality for facility residents.

Digital health implementations and investments continue to expand. As the reliance on digital health increases, it is imperative to implement technologies with inclusive and accessible approaches. A conceptual model can be used to guide equity-focused digital health implementations to improve suitability and uptake in diverse populations. The objective of this study is expand an implementation model with recommendations on the equitable implementation of new digital health technologies. The Digital Health Equity-Focused Implementation Research (DH-EquIR) conceptual model was developed based on a rigorous review of digital health implementation and health equity literature. The Equity-Focused Implementation Research for Health Programs (EquIR) model was used as a starting point and merged with digital equity and digital health implementation models. Existing theoretical frameworks and models were appraised as well as individual equity-sensitive implementation studies. Patient and program-related concepts related to digital equity, digital health implementation, and assessment of social/digital determinants of health were included. Sixty-two articles were analyzed to inform the adaption of the EquIR model for digital health. These articles included digital health equity models and frameworks, digital health implementation models and frameworks, research articles, guidelines, and concept analyses. Concepts were organized into EquIR conceptual groupings, including population health status, planning the program, designing the program, implementing the program, and equity-focused implementation outcomes. The adapted DH-EquIR conceptual model diagram was created as well as detailed tables displaying related equity concepts, evidence gaps in source articles, and analysis of existing equity-related models and tools. The DH-EquIR model serves to guide digital health developers and implementation specialists to promote the inclusion of health-equity planning in every phase of implementation. In addition, it can assist researchers and product developers to avoid repeating the mistakes that have led to inequities in the implementation of digital health across populations.