Searched for: in-biosketch:yes
person:aphiny01
Development, implementation, and prospective validation of a model to predict 60-day end-of-life in hospitalized adults upon admission at three sites
Major, Vincent J; Aphinyanaphongs, Yindalon
BACKGROUND:Automated systems that use machine learning to estimate a patient's risk of death are being developed to influence care. There remains sparse transparent reporting of model generalizability in different subpopulations especially for implemented systems. METHODS:A prognostic study included adult admissions at a multi-site, academic medical center between 2015 and 2017. A predictive model for all-cause mortality (including initiation of hospice care) within 60 days of admission was developed. Model generalizability is assessed in temporal validation in the context of potential demographic bias. A subsequent prospective cohort study was conducted at the same sites between October 2018 and June 2019. Model performance during prospective validation was quantified with areas under the receiver operating characteristic and precision recall curves stratified by site. Prospective results include timeliness, positive predictive value, and the number of actionable predictions. RESULTS:Three years of development data included 128,941 inpatient admissions (94,733 unique patients) across sites where patients are mostly white (61%) and female (60%) and 4.2% led to death within 60 days. A random forest model incorporating 9614 predictors produced areas under the receiver operating characteristic and precision recall curves of 87.2 (95% CI, 86.1-88.2) and 28.0 (95% CI, 25.0-31.0) in temporal validation. Performance marginally diverges within sites as the patient mix shifts from development to validation (patients of one site increases from 10 to 38%). Applied prospectively for nine months, 41,728 predictions were generated in real-time (median [IQR], 1.3 [0.9, 32] minutes). An operating criterion of 75% positive predictive value identified 104 predictions at very high risk (0.25%) where 65% (50 from 77 well-timed predictions) led to death within 60 days. CONCLUSION/CONCLUSIONS:Temporal validation demonstrates good model discrimination for 60-day mortality. Slight performance variations are observed across demographic subpopulations. The model was implemented prospectively and successfully produced meaningful estimates of risk within minutes of admission.
PMID: 32894128
ISSN: 1472-6947
CID: 4588762
Prevalence and Outcomes of D-Dimer Elevation in Hospitalized Patients With COVID-19
Berger, Jeffrey S; Kunichoff, Dennis; Adhikari, Samrachana; Ahuja, Tania; Amoroso, Nancy; Aphinyanaphongs, Yindalon; Cao, Meng; Goldenberg, Ronald; Hindenburg, Alexander; Horowitz, James; Parnia, Sam; Petrilli, Christopher; Reynolds, Harmony; Simon, Emma; Slater, James; Yaghi, Shadi; Yuriditsky, Eugene; Hochman, Judith; Horwitz, Leora I
OBJECTIVE:<0.001). Rates of adverse events increased with the magnitude of D-dimer elevation; individuals with presenting D-dimer >2000 ng/mL had the highest risk of critical illness (66%), thrombotic event (37.8%), acute kidney injury (58.3%), and death (47%). CONCLUSIONS:Abnormal D-dimer was frequently observed at admission with COVID-19 and was associated with higher incidence of critical illness, thrombotic events, acute kidney injury, and death. The optimal management of patients with elevated D-dimer in COVID-19 requires further study.
PMID: 32840379
ISSN: 1524-4636
CID: 4574192
Thrombosis in Hospitalized Patients With COVID-19 in a New York City Health System
Bilaloglu, Seda; Aphinyanaphongs, Yin; Jones, Simon; Iturrate, Eduardo; Hochman, Judith; Berger, Jeffrey S
PMCID:7372509
PMID: 32702090
ISSN: 1538-3598
CID: 4532682
Estimating real-world performance of a predictive model: a case-study in predicting mortality
Major, Vincent J; Jethani, Neil; Aphinyanaphongs, Yindalon
Objective/UNASSIGNED:One primary consideration when developing predictive models is downstream effects on future model performance. We conduct experiments to quantify the effects of experimental design choices, namely cohort selection and internal validation methods, on (estimated) real-world model performance. Materials and Methods/UNASSIGNED:Four years of hospitalizations are used to develop a 1-year mortality prediction model (composite of death or initiation of hospice care). Two common methods to select appropriate patient visits from their encounter history (backwards-from-outcome and forwards-from-admission) are combined with 2 testing cohorts (random and temporal validation). Two models are trained under otherwise identical conditions, and their performances compared. Operating thresholds are selected in each test set and applied to a "real-world" cohort of labeled admissions from another, unused year. Results/UNASSIGNED: = 92 148). Both selection methods produce similar performances when applied to a random test set. However, when applied to the temporally defined "real-world" set, forwards-from-admission yields higher areas under the ROC and precision recall curves (88.3% and 56.5% vs. 83.2% and 41.6%). Discussion/UNASSIGNED:A backwards-from-outcome experiment manipulates raw training data, simplifying the experiment. This manipulated data no longer resembles real-world data, resulting in optimistic estimates of test set performance, especially at high precision. In contrast, a forwards-from-admission experiment with a temporally separated test set consistently and conservatively estimates real-world performance. Conclusion/UNASSIGNED:Experimental design choices impose bias upon selected cohorts. A forwards-from-admission experiment, validated temporally, can conservatively estimate real-world performance. LAY SUMMARY/UNASSIGNED:The routine care of patients stands to benefit greatly from assistive technologies, including data-driven risk assessment. Already, many different machine learning and artificial intelligence applications are being developed from complex electronic health record data. To overcome challenges that arise from such data, researchers often start with simple experimental approaches to test their work. One key component is how patients (and their healthcare visits) are selected for the study from the pool of all patients seen. Another is how the group of patients used to create the risk estimator differs from the group used to evaluate how well it works. These choices complicate how the experimental setting compares to the real-world application to patients. For example, different selection approaches that depend on each patient's future outcome can simplify the experiment but are impractical upon implementation as these data are unavailable. We show that this kind of "backwards" experiment optimistically estimates how well the model performs. Instead, our results advocate for experiments that select patients in a "forwards" manner and "temporal" validation that approximates training on past data and implementing on future data. More robust results help gauge the clinical utility of recent works and aid decision-making before implementation into practice.
PMCID:7382635
PMID: 32734165
ISSN: 2574-2531
CID: 4540712
Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination
Chen, Ji; Chokshi, Sara; Hegde, Roshini; Gonzalez, Javier; Iturrate, Eduardo; Aphinyanaphongs, Yin; Mann, Devin
BACKGROUND:Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)-integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician's interaction with these alerts in general. OBJECTIVE:This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the signal of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts. METHODS:We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician's interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients. RESULTS:During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; P=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; P=.20). CONCLUSIONS:All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.
PMID: 32347813
ISSN: 1438-8871
CID: 4412352
Detecting illicit opioid content on Twitter
Tofighi, Babak; Aphinyanaphongs, Yindalon; Marini, Christina; Ghassemlou, Shouron; Nayebvali, Peyman; Metzger, Isabel; Raghunath, Ananditha; Thomas, Shailin
INTRODUCTION AND AIMS/OBJECTIVE:This article examines the feasibility of leveraging Twitter to detect posts authored by people who use opioids (PWUO) or content related to opioid use disorder (OUD), and manually develop a multidimensional taxonomy of relevant tweets. DESIGN AND METHODS/METHODS:Twitter messages were collected between June and October 2017 (n = 23 827) and evaluated using an inductive coding approach. Content was then manually classified into two axes (n = 17 420): (i) user experience regarding accessing, using, or recovery from illicit opioids; and (ii) content categories (e.g. policies, medical information, jokes/sarcasm). RESULTS:The most prevalent categories consisted of jokes or sarcastic comments pertaining to OUD, PWUOs or hypothetically using illicit opioids (63%), informational content about treatments for OUD, overdose prevention or accessing self-help groups (20%), and commentary about government opioid policy or news related to opioids (17%). Posts by PWUOs centered on identifying illicit sources for procuring opioids (i.e. online, drug dealers; 49%), symptoms and/or strategies to quell opioid withdrawal symptoms (21%), and combining illicit opioid use with other substances, such as cocaine or benzodiazepines (17%). State and public health experts infrequently posted content pertaining to OUD (1%). DISCUSSION AND CONCLUSIONS/CONCLUSIONS:Twitter offers a feasible approach to identify PWUO. Further research is needed to evaluate the efficacy of Twitter to disseminate evidence-based content and facilitate linkage to treatment and harm reduction services.
PMID: 32202005
ISSN: 1465-3362
CID: 4357472
Electronic Cigarette Aerosol Modulates the Oral Microbiome and Increases Risk of Infection
Pushalkar, Smruti; Paul, Bidisha; Li, Qianhao; Yang, Jian; Vasconcelos, Rebeca; Makwana, Shreya; González, Juan Muñoz; Shah, Shivm; Xie, Chengzhi; Janal, Malvin N; Queiroz, Erica; Bederoff, Maria; Leinwand, Joshua; Solarewicz, Julia; Xu, Fangxi; Aboseria, Eman; Guo, Yuqi; Aguallo, Deanna; Gomez, Claudia; Kamer, Angela; Shelley, Donna; Aphinyanaphongs, Yindalon; Barber, Cheryl; Gordon, Terry; Corby, Patricia; Li, Xin; Saxena, Deepak
The trend of e-cigarette use among teens is ever increasing. Here we show the dysbiotic oral microbial ecology in e-cigarette users influencing the local host immune environment compared with non-smoker controls and cigarette smokers. Using 16S rRNA high-throughput sequencing, we evaluated 119 human participants, 40 in each of the three cohorts, and found significantly altered beta-diversity in e-cigarette users (p = 0.006) when compared with never smokers or tobacco cigarette smokers. The abundance of Porphyromonas and Veillonella (p = 0.008) was higher among vapers. Interleukin (IL)-6 and IL-1β were highly elevated in e-cigarette users when compared with non-users. Epithelial cell-exposed e-cigarette aerosols were more susceptible for infection. In vitro infection model of premalignant Leuk-1 and malignant cell lines exposed to e-cigarette aerosol and challenged by Porphyromonas gingivalis and Fusobacterium nucleatum resulted in elevated inflammatory response. Our findings for the first time demonstrate that e-cigarette users are more prone to infection.
PMID: 32105635
ISSN: 2589-0042
CID: 4323572
A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients
Razavian, Narges; Major, Vincent J; Sudarshan, Mukund; Burk-Rafel, Jesse; Stella, Peter; Randhawa, Hardev; Bilaloglu, Seda; Chen, Ji; Nguy, Vuthy; Wang, Walter; Zhang, Hao; Reinstein, Ilan; Kudlowitz, David; Zenger, Cameron; Cao, Meng; Zhang, Ruina; Dogra, Siddhant; Harish, Keerthi B; Bosworth, Brian; Francois, Fritz; Horwitz, Leora I; Ranganath, Rajesh; Austrian, Jonathan; Aphinyanaphongs, Yindalon
The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.
PMCID:7538971
PMID: 33083565
ISSN: 2398-6352
CID: 4640992
Performance Evaluation of A Machine Learning Model For Systematic Identification of Wild-type Transthyretin Amyloid Cardiomyopathy At Two Academic Medical Centers [Meeting Abstract]
Heitner, Stephen; Elman, Miriam R.; Masri, Ahmad; Aphinyanaphongs, Yindalon; Reyentovich, Alex; Ateya, Mohammad; Emir, Birol; Fowler, Ryan; Mills, J. Rebecca; Nolen, Kim D.; Sohn, Alexis; Huda, Ahsan; Castano, Adam; Bruno, Marianna
ISI:000579889600095
ISSN: 1071-9164
CID: 4677572
Challenges in translating mortality risk to the point of care [Editorial]
Major, Vincent J; Aphinyanaphongs, Yindalon
PMID: 31481481
ISSN: 2044-5423
CID: 4067212