Searched for: in-biosketch:yes
person:aphiny01
Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback
Schaye, Verity; Miller, Louis; Kudlowitz, David; Chun, Jonathan; Burk-Rafel, Jesse; Cocks, Patrick; Guzman, Benedict; Aphinyanaphongs, Yindalon; Marin, Marina
BACKGROUND:Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability. OBJECTIVE:Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool. DESIGN, PARTICIPANTS, AND MAIN MEASURES/UNASSIGNED:The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting. KEY RESULTS/RESULTS:The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality. CONCLUSIONS:The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.
PMID: 33945113
ISSN: 1525-1497
CID: 4866222
AUTOMATED MACHINE LEARNING WITH AUTOGLUON TO PREDICT POSTOPERATIVE PNEUMONIA USING THE AMERICAN COLLEGE OF SURGEONS' NATIONAL SURGICAL QUALITY IMPROVEMENT PROGRAM DATABASE [Meeting Abstract]
Brill, Kenneth; Thangirala, Avinash; Aphinyanaphongs, Yin; Chen, Ji; Hu, Eric; Kelleher, Andrew C.; Martin, Jacob; Oeding, Jacob F.; Ostberg, Nicolai; Katz, Gregory; Brejt, Shelly; Ramanathan, Rudra; Kan, Karen
ISI:000895468902427
ISSN: 0012-3692
CID: 5522432
Quantitative and Qualitative Evaluation of Provider Use of a Novel Machine Learning Model for Favorable Outcome Prediction
Yang, Elisabeth; Aphinyanaphongs, Yin; Punjabi, Paawan V; Austrian, Jonathan; Wiesenfeld, Batia
Predictive models may be particularly beneficial to clinicians when they face uncertainty and seek to develop a mental model of disease progression, but we know little about the post-implementation effects of predictive models on clinicians' experience of their work. Combining survey and interview methods, we found that providers using a predictive algorithm reported being significantly less uncertain and better able to anticipate, plan and prepare for patient discharge than non-users. The tool helped hospitalists form and develop confidence in their mental models of a novel disease (Covid-19). Yet providers' attention to the predictive tool declined as their confidence in their own mental models grew. Predictive algorithms that not only offer data but also provide feedback on decisions, thus supporting providers' motivation for continuous learning, hold promise for more sustained provider attention and cognition augmentation.
PMCID:10148285
PMID: 37128409
ISSN: 1942-597x
CID: 5542392
Platelets contribute to disease severity in COVID-19
Barrett, Tessa J; Bilaloglu, Seda; Cornwell, Macintosh; Burgess, Hannah M; Virginio, Vitor W; Drenkova, Kamelia; Ibrahim, Homam; Yuriditsky, Eugene; Aphinyanaphongs, Yin; Lifshitz, Mark; Xia Liang, Feng; Alejo, Julie; Smith, Grace; Pittaluga, Stefania; Rapkiewicz, Amy V; Wang, Jun; Iancu-Rubin, Camelia; Mohr, Ian; Ruggles, Kelly; Stapleford, Kenneth A; Hochman, Judith; Berger, Jeffrey S
OBJECTIVE:Heightened inflammation, dysregulated immunity, and thrombotic events are characteristic of hospitalized COVID-19 patients. Given that platelets are key regulators of thrombosis, inflammation, and immunity they represent prime candidates as mediators of COVID-19-associated pathogenesis. The objective of this study was to understand the contribution of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) to the platelet phenotype via phenotypic (activation, aggregation) and transcriptomic characterization. APPROACH AND RESULTS/UNASSIGNED:In a cohort of 3915Â hospitalized COVID-19 patients, we analyzed blood platelet indices collected at hospital admission. Following adjustment for demographics, clinical risk factors, medication, and biomarkers of inflammation and thrombosis, we find platelet count, size, and immaturity are associated with increased critical illness and all-cause mortality. Bone marrow, lung tissue, and blood from COVID-19 patients revealed the presence of SARS-CoV-2 virions in megakaryocytes and platelets. Characterization of COVID-19 platelets found them to be hyperreactive (increased aggregation, and expression of P-selectin and CD40) and to have a distinct transcriptomic profile characteristic of prothrombotic large and immature platelets. In vitro mechanistic studies highlight that the interaction of SARS-CoV-2 with megakaryocytes alters the platelet transcriptome, and its effects are distinct from the coronavirus responsible for the common cold (CoV-OC43). CONCLUSIONS:Platelet count, size, and maturity associate with increased critical illness and all-cause mortality among hospitalized COVID-19 patients. Profiling tissues and blood from COVID-19 patients revealed that SARS-CoV-2 virions enter megakaryocytes and platelets and associate with alterations to the platelet transcriptome and activation profile.
PMID: 34538015
ISSN: 1538-7836
CID: 5018172
Medication utilization among vascular dementia population
Razavian, Narges; Dodson, John; Masurkar, Arjun V; Wisniewski, Thomas; Horwitz, Leora; Aphinyanaphongs, Yindalon
BACKGROUND:It is estimated that up to 40% of Alzheimer's Disease and Related Dementias cases can be prevented or delayed by addressing modifiable factors including those that influence vascular risk (hypertension, obesity, smoking, physical activity, diabetes). Prevention may be particularly important in the vascular dementia subtypes. Despite the supporting evidence, the rates of medical therapy to reduce vascular risk are not well described. METHOD/METHODS:We assessed the utilization of statins, aspirin, and blood pressure (BP) medications in adults age ≥65 years cared for at NYU Langone Health, as recorded in the electronic health record. We included two cohorts: cohort 1 included patients who were diagnosed with vascular dementia (VaD) at NYU Langone Barlow Center for Memory Evaluation between January 1, 2015 and June 24, 2019. Cohort 2 extended the inclusion to seniors with VD diagnosis by any NYU Langone physician. Definitions for vascular dementia, the covariates assessed, and medications that we included in each category are shown in Tables 1-3. RESULT/RESULTS:We included 419 and 3745 patients in cohort 1 and cohort 2, respectively. Table 4 shows the characteristics and medication adherence in cohorts 1 and 2. In cohort 1, the prescription rates for statins, aspirin, and BP medications were 66%, 66%, 70%. In cohort 2, the rates for statin, aspirin, and BP medications were 56%, 46%, and 65%, respectively. The differences between prescription rates in cohort 1 and 2 for the three medication groups were statistically significant (p<0.05). CONCLUSION/CONCLUSIONS:Our analysis of the utilization of cardiovascular medications among patients with vascular dementia illuminates potential gaps both among patients who receive care at specialty clinics, as well as the overall population with vascular dementia. The rates of medication utilization are higher for patients under the care of cognitive neurologists. Electronic health records can help identify large cohorts of patients who may benefit from improved access to preventative measures including cardiovascular medications.
PMID: 34971267
ISSN: 1552-5279
CID: 5108332
Development and Validation of a Machine Learning-Based Decision Support Tool for Residency Applicant Screening and Review
Burk-Rafel, Jesse; Reinstein, Ilan; Feng, James; Kim, Moosun Brad; Miller, Louis H; Cocks, Patrick M; Marin, Marina; Aphinyanaphongs, Yindalon
PURPOSE:Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review. METHOD:Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations. RESULTS:The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review. CONCLUSIONS:The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods.
PMID: 34348383
ISSN: 1938-808x
CID: 5050022
Validation of parsimonious prognostic models for patients infected with COVID-19
Harish, Keerthi; Zhang, Ben; Stella, Peter; Hauck, Kevin; Moussa, Marwa M; Adler, Nicole M; Horwitz, Leora I; Aphinyanaphongs, Yindalon
OBJECTIVES/OBJECTIVE:Predictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data. METHODS:We performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020. RESULTS:Most models failed validation when applied to our institution's data. Included studies reported an average validation area under the receiver-operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies' reported AUROC values. DISCUSSION/CONCLUSIONS:Published and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations. CONCLUSIONS:Clinicians should employ caution when applying models for clinical prediction without careful validation on local data.
PMCID:8421114
PMID: 34479962
ISSN: 2632-1009
CID: 5000192
Predicting inpatient pharmacy order interventions using provider action data
Balestra, Martina; Chen, Ji; Iturrate, Eduardo; Aphinyanaphongs, Yindalon; Nov, Oded
Objective/UNASSIGNED:The widespread deployment of electronic health records (EHRs) has introduced new sources of error and inefficiencies to the process of ordering medications in the hospital setting. Existing work identifies orders that require pharmacy intervention by comparing them to a patient's medical records. In this work, we develop a machine learning model for identifying medication orders requiring intervention using only provider behavior and other contextual features that may reflect these new sources of inefficiencies. Materials and Methods/UNASSIGNED:Data on providers' actions in the EHR system and pharmacy orders were collected over a 2-week period in a major metropolitan hospital system. A classification model was then built to identify orders requiring pharmacist intervention. We tune the model to the context in which it would be deployed and evaluate global and local feature importance. Results/UNASSIGNED:The resultant model had an area under the receiver-operator characteristic curve of 0.91 and an area under the precision-recall curve of 0.44. Conclusions/UNASSIGNED:Providers' actions can serve as useful predictors in identifying medication orders that require pharmacy intervention. Careful model tuning for the clinical context in which the model is deployed can help to create an effective tool for improving health outcomes without using sensitive patient data.
PMCID:8490931
PMID: 34617009
ISSN: 2574-2531
CID: 5092072
An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department
Shamout, Farah E; Shen, Yiqiu; Wu, Nan; Kaku, Aakash; Park, Jungkyu; Makino, Taro; Jastrzębski, Stanisław; Witowski, Jan; Wang, Duo; Zhang, Ben; Dogra, Siddhant; Cao, Meng; Razavian, Narges; Kudlowitz, David; Azour, Lea; Moore, William; Lui, Yvonne W; Aphinyanaphongs, Yindalon; Fernandez-Granda, Carlos; Geras, Krzysztof J
During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.
PMID: 33980980
ISSN: 2398-6352
CID: 4867572
Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations
Jethani, Neil; Sudarshan, Mukund; Aphinyanaphongs, Yindalon; Ranganath, Rajesh
While the need for interpretable machine learning has been established, many common approaches are slow, lack fidelity, or hard to evaluate. Amortized explanation methods reduce the cost of providing interpretations by learning a global selector model that returns feature importances for a single instance of data. The selector model is trained to optimize the fidelity of the interpretations, as evaluated by a predictor model for the target. Popular methods learn the selector and predictor model in concert, which we show allows predictions to be encoded within interpretations. We introduce EVAL-X as a method to quantitatively evaluate interpretations and REAL-X as an amortized explanation method, which learn a predictor model that approximates the true data generating distribution given any subset of the input. We show EVAL-X can detect when predictions are encoded in interpretations and show the advantages of REAL-X through quantitative and radiologist evaluation.
PMCID:8096519
PMID: 33954293
ISSN: 2640-3498
CID: 4866542