Faculty Bibliography

BACKGROUND:It is estimated that up to 40% of Alzheimer's Disease and Related Dementias cases can be prevented or delayed by addressing modifiable factors including those that influence vascular risk (hypertension, obesity, smoking, physical activity, diabetes). Prevention may be particularly important in the vascular dementia subtypes. Despite the supporting evidence, the rates of medical therapy to reduce vascular risk are not well described. METHOD/METHODS:We assessed the utilization of statins, aspirin, and blood pressure (BP) medications in adults age ≥65 years cared for at NYU Langone Health, as recorded in the electronic health record. We included two cohorts: cohort 1 included patients who were diagnosed with vascular dementia (VaD) at NYU Langone Barlow Center for Memory Evaluation between January 1, 2015 and June 24, 2019. Cohort 2 extended the inclusion to seniors with VD diagnosis by any NYU Langone physician. Definitions for vascular dementia, the covariates assessed, and medications that we included in each category are shown in Tables 1-3. RESULT/RESULTS:We included 419 and 3745 patients in cohort 1 and cohort 2, respectively. Table 4 shows the characteristics and medication adherence in cohorts 1 and 2. In cohort 1, the prescription rates for statins, aspirin, and BP medications were 66%, 66%, 70%. In cohort 2, the rates for statin, aspirin, and BP medications were 56%, 46%, and 65%, respectively. The differences between prescription rates in cohort 1 and 2 for the three medication groups were statistically significant (p<0.05). CONCLUSION/CONCLUSIONS:Our analysis of the utilization of cardiovascular medications among patients with vascular dementia illuminates potential gaps both among patients who receive care at specialty clinics, as well as the overall population with vascular dementia. The rates of medication utilization are higher for patients under the care of cognitive neurologists. Electronic health records can help identify large cohorts of patients who may benefit from improved access to preventative measures including cardiovascular medications.

PURPOSE:Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review. METHOD:Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations. RESULTS:The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review. CONCLUSIONS:The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods.

OBJECTIVES/OBJECTIVE:Predictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data. METHODS:We performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020. RESULTS:Most models failed validation when applied to our institution's data. Included studies reported an average validation area under the receiver-operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies' reported AUROC values. DISCUSSION/CONCLUSIONS:Published and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations. CONCLUSIONS:Clinicians should employ caution when applying models for clinical prediction without careful validation on local data.

Objective/UNASSIGNED:The widespread deployment of electronic health records (EHRs) has introduced new sources of error and inefficiencies to the process of ordering medications in the hospital setting. Existing work identifies orders that require pharmacy intervention by comparing them to a patient's medical records. In this work, we develop a machine learning model for identifying medication orders requiring intervention using only provider behavior and other contextual features that may reflect these new sources of inefficiencies. Materials and Methods/UNASSIGNED:Data on providers' actions in the EHR system and pharmacy orders were collected over a 2-week period in a major metropolitan hospital system. A classification model was then built to identify orders requiring pharmacist intervention. We tune the model to the context in which it would be deployed and evaluate global and local feature importance. Results/UNASSIGNED:The resultant model had an area under the receiver-operator characteristic curve of 0.91 and an area under the precision-recall curve of 0.44. Conclusions/UNASSIGNED:Providers' actions can serve as useful predictors in identifying medication orders that require pharmacy intervention. Careful model tuning for the clinical context in which the model is deployed can help to create an effective tool for improving health outcomes without using sensitive patient data.

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

While the need for interpretable machine learning has been established, many common approaches are slow, lack fidelity, or hard to evaluate. Amortized explanation methods reduce the cost of providing interpretations by learning a global selector model that returns feature importances for a single instance of data. The selector model is trained to optimize the fidelity of the interpretations, as evaluated by a predictor model for the target. Popular methods learn the selector and predictor model in concert, which we show allows predictions to be encoded within interpretations. We introduce EVAL-X as a method to quantitatively evaluate interpretations and REAL-X as an amortized explanation method, which learn a predictor model that approximates the true data generating distribution given any subset of the input. We show EVAL-X can detect when predictions are encoded in interpretations and show the advantages of REAL-X through quantitative and radiologist evaluation.

The transformation of patient-clinician relationships with AI-based medical advice is discussed. many new tools are based on entirely new "˜black-box"™ AI-based technologies, whose inner workings are likely not fully understood by patients or clinicians. Most patients with Type 1 diabetes now use continuous glucose monitors and insulin pumps to tightly manage their disease. Their clinicians carefully review the data streams from both devices to recommend dosage adjustments. Recently new automated recommender systems to monitor and analyze food intake, insulin doses, physical activity, and other factors influencing glucose levels, and provide data-intensive, AI-based recommendations on how to titrate the regimen, are in different stages of FDA approval using "˜black box"™ technology, which is an alluring proposition for a clinical scenario that requires identification of meaningful patterns in complex and voluminous data.

The rapid spread of COVID-19 cases in recent months has strained hospital resources, making rapid and accurate triage of patients presenting to emergency departments a necessity. Machine learning techniques using clinical data such as chest X-rays have been used to predict which patients are most at risk of deterioration. We consider the task of predicting two types of patient deterioration based on chest X-rays: adverse event deterioration (i.e., transfer to the intensive care unit, intubation, or mortality) and increased oxygen requirements beyond 6 L per day. Due to the relative scarcity of COVID-19 patient data, existing solutions leverage supervised pretraining on related non-COVID images, but this is limited by the differences between the pretraining data and the target COVID-19 patient data. In this paper, we use self-supervised learning based on the momentum contrast (MoCo) method in the pretraining phase to learn more general image representations to use for downstream tasks. We present three results. The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0.742 for predicting an adverse event within 96 hours (compared to 0.703 with supervised pretraining) and an AUC of 0.765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0.749 with supervised pretraining). We then propose a new transformer-based architecture that can process sequences of multiple images for prediction and show that this model can achieve an improved AUC of 0.786 for predicting an adverse event at 96 hours and an AUC of 0.848 for predicting mortalities at 96 hours. A small pilot clinical study suggested that the prediction accuracy of our model is comparable to that of experienced radiologists analyzing the same information.