Faculty Bibliography

TOPIC/UNASSIGNED:This scoping review summarizes artificial intelligence (AI) reporting in ophthalmology literature in respect to model development and validation. We characterize the state of transparency in reporting of studies prospectively validating models for disease classification. CLINICAL RELEVANCE/UNASSIGNED:Understanding what elements authors currently describe regarding their AI models may aid in the future standardization of reporting. This review highlights the need for transparency to facilitate the critical appraisal of models prior to clinical implementation, to minimize bias and inappropriate use. Transparent reporting can improve effective and equitable use in clinical settings. METHODS/UNASSIGNED:Eligible articles (as of January 2022) from PubMed, Embase, Web of Science, and CINAHL were independently screened by 2 reviewers. All observational and clinical trial studies evaluating the performance of an AI model for disease classification of ophthalmic conditions were included. Studies were evaluated for reporting of parameters derived from reporting guidelines (CONSORT-AI, MI-CLAIM) and our previously published editorial on model cards. The reporting of these factors, which included basic model and dataset details (source, demographics), and prospective validation outcomes, were summarized. RESULTS/UNASSIGNED:Thirty-seven prospective validation studies were included in the scoping review. Eleven additional associated training and/or retrospective validation studies were included if this information could not be determined from the primary articles. These 37 studies validated 27 unique AI models; multiple studies evaluated the same algorithms (EyeArt, IDx-DR, and Medios AI). Details of model development were variably reported; 18 of 27 models described training dataset annotation and 10 of 27 studies reported training data distribution. Demographic information of training data was rarely reported; 7 of the 27 unique models reported age and gender and only 2 reported race and/or ethnicity. At the level of prospective clinical validation, age and gender of populations was more consistently reported (29 and 28 of 37 studies, respectively), but only 9 studies reported race and/or ethnicity data. Scope of use was difficult to discern for the majority of models. Fifteen studies did not state or imply primary users. CONCLUSION/UNASSIGNED:Our scoping review demonstrates variable reporting of information related to both model development and validation. The intention of our study was not to assess the quality of the factors we examined, but to characterize what information is, and is not, regularly reported. Our results suggest the need for greater transparency in the reporting of information necessary to determine the appropriateness and fairness of these tools prior to clinical use. FINANCIAL DISCLOSURES/UNASSIGNED:Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

BACKGROUND AND OBJECTIVES/OBJECTIVE:Clinical registries are critical for modern surgery and underpin outcomes research, device monitoring, and trial development. However, existing approaches to registry construction are labor-intensive, costly, and prone to manual error. Natural language processing techniques combined with electronic health record (EHR) data sets can theoretically automate the construction and maintenance of registries. Our aim was to automate the generation of a spine surgery registry at an academic medical center using regular expression (regex) classifiers developed by neurosurgeons to combine domain expertise with interpretable algorithms. METHODS:We used a Hadoop data lake consisting of all the information generated by an academic medical center. Using this database and structured query language queries, we retrieved every operative note written in the department of neurosurgery since our transition to EHR. Notes were parsed using regex classifiers and compared with a random subset of 100 manually reviewed notes. RESULTS:A total of 31 502 operative cases were downloaded and processed using regex classifiers. The codebase required 5 days of development, 3 weeks of validation, and less than 1 hour for the software to generate the autoregistry. Regex classifiers had an average accuracy of 98.86% at identifying both spinal procedures and the relevant vertebral levels, and it correctly identified the entire list of defined surgical procedures in 89% of patients. We were able to identify patients who required additional operations within 30 days to monitor outcomes and quality metrics. CONCLUSION/CONCLUSIONS:This study demonstrates the feasibility of automatically generating a spine registry using the EHR and an interpretable, customizable natural language processing algorithm which may reduce pitfalls associated with manual registry development and facilitate rapid clinical research.

BACKGROUND AND OBJECTIVES/OBJECTIVE:Advances in targeted therapies and wider application of stereotactic radiosurgery (SRS) have redefined outcomes of patients with brain metastases. Under modern treatment paradigms, there remains limited characterization of which aspects of disease drive demise and in what frequencies. This study aims to characterize the primary causes of terminal decline and evaluate differences in underlying intracranial tumor dynamics in patients with metastatic brain cancer. These fundamental details may help guide management, patient counseling, and research priorities. METHODS:Using NYUMets-Brain-the largest, longitudinal, real-world, open data set of patients with brain metastases-patients treated at New York University Langone Health between 2012 and 2021 with SRS were evaluated. A review of electronic health records allowed for the determination of a primary cause of death in patients who died during the study period. Causes were classified in mutually exclusive, but collectively exhaustive, categories. Multilevel models evaluated for differences in dynamics of intracranial tumors, including changes in volume and number. RESULTS:Of 439 patients with end-of-life data, 73.1% died secondary to systemic disease, 10.3% died secondary to central nervous system (CNS) disease, and 16.6% died because of other causes. CNS deaths were driven by acute increases in intracranial pressure (11%), development of focal neurological deficits (18%), treatment-resistant seizures (11%), and global decline driven by increased intracranial tumor burden (60%). Rate of influx of new intracranial tumors was almost twice as high in patients who died compared with those who survived (P < .001), but there was no difference in rates of volume change per intracranial tumor (P = .95). CONCLUSION/CONCLUSIONS:Most patients with brain metastases die secondary to systemic disease progression. For patients who die because of neurological disease, tumor dynamics and cause of death mechanisms indicate that the primary driver of decline for many may be unchecked systemic disease with unrelenting spread of new tumors to the CNS rather than failure of local growth control.

Over the past generation, outcome measures in spine care have evolved from a reliance on clinician-reported assessment toward recognizing the importance of the patient's perspective and the wide incorporation of patient-reported outcomes (PROs). While patient-reported outcomes are now considered an integral component of outcomes assessments, they cannot wholly capture the state of a patient's functionality. There is a clear need for quantitative and objective patient-centered outcome measures. The pervasiveness of smartphones and wearable devices in modern society, which passively collect data related to health, has ushered in a new era of spine care outcome measurement. The patterns emerging from these data, so-called "digital biomarkers," can accurately describe characteristics of a patient's health, disease, or recovery state. Broadly, the spine care community has thus far concentrated on digital biomarkers related to mobility, although the researcher's toolkit is anticipated to expand in concert with advancements in technology. In this review of the nascent literature, we describe the evolution of spine care outcome measurements, outline how digital biomarkers can supplement current clinician-driven and patient-driven measures, appraise the present and future of the field in the modern era, as well as discuss present limitations and areas for further study, with a focus on smartphones (see Supplemental Digital Content , http://links.lww.com/NEU/D809 , for a similar appraisal of wearable devices).

Physicians make critical time-constrained decisions every day. Clinical predictive models can help physicians and administrators make decisions by forecasting clinical and operational events. Existing structured data-based clinical predictive models have limited use in everyday practice owing to complexity in data processing, as well as model development and deployment^1-3. Here we show that unstructured clinical notes from the electronic health record can enable the training of clinical language models, which can be used as all-purpose clinical predictive engines with low-resistance development and deployment. Our approach leverages recent advances in natural language processing^4,5 to train a large language model for medical language (NYUTron) and subsequently fine-tune it across a wide range of clinical and operational predictive tasks. We evaluated our approach within our health system for five such tasks: 30-day all-cause readmission prediction, in-hospital mortality prediction, comorbidity index prediction, length of stay prediction, and insurance denial prediction. We show that NYUTron has an area under the curve (AUC) of 78.7-94.9%, with an improvement of 5.36-14.7% in the AUC compared with traditional models. We additionally demonstrate the benefits of pretraining with clinical text, the potential for increasing generalizability to different sites through fine-tuning and the full deployment of our system in a prospective, single-arm trial. These results show the potential for using clinical language models in medicine to read alongside physicians and provide guidance at the point of care.

Artificial intelligence and machine learning applications are becoming increasingly popular in health care and medical devices. The development of accurate machine learning algorithms requires large quantities of good and diverse data. This poses a challenge in health care because of the sensitive nature of sharing patient data. Decentralized algorithms through federated learning avoid data aggregation. In this paper we give an overview of federated learning, current examples in healthcare and ophthalmology, challenges, and next steps.

BACKGROUND:Spine abnormalities are a common manifestation of Neurofibromatosis Type 1 (NF1); however, the outcomes of surgical treatment for NF1-associated spinal deformity are not well explored. The purpose of this study was to investigate the outcome and risk profiles of multilevel fusion surgery for NF1 patients. METHODS:The National Inpatient Sample was queried for NF1 and non-NF1 patient populations with neuromuscular scoliosis who underwent multilevel fusion surgery involving eight or more vertebral levels between 2004 and 2017. Multivariate regression modeling was used to explore the relationship between perioperative variables and pertinent outcomes. RESULTS:Of the 55,485 patients with scoliosis, 533 patients (0.96%) had NF1. Patients with NF1 were more likely to have comorbid solid tumors (P < 0.0001), clinical depression (P < 0.0001), peripheral vascular disease (P < 0.0001), and hypertension (P < 0.001). Following surgery, NF1 patients had a higher incidence of hydrocephalus (0.6% vs. 1.9% P = 0.002), seizures (4.9% vs. 5.7% P = 0.006), and accidental vessel laceration (0.3% vs.1.9% P = 0.011). Although there were no differences in overall complication rates or in-hospital mortality, multivariate regression revealed NF1 patients had an increased probability of pulmonary (OR 0.5, 95%CI 0.3-0.8, P = 0.004) complications. There were no significant differences in utilization, including nonhome discharge or extended hospitalization; however, patients with NF1 had higher total hospital charges (mean -$18739, SE 3384, P < 0.0001). CONCLUSIONS:These findings indicate that NF1 is associated with certain complications following multilevel fusion surgery but does not appear to be associated with differences in quality or cost outcomes. These results provide some guidance to surgeons and other healthcare professionals in their perioperative decision making by raising awareness about risk factors for NF1 patients undergoing multilevel fusion surgery. We intend for this study to set the national baseline for complications after multilevel fusion in the NF1 population.

BACKGROUND:The development of accurate machine learning algorithms requires sufficient quantities of diverse data. This poses a challenge in health care because of the sensitive and siloed nature of biomedical information. Decentralized algorithms through federated learning (FL) avoid data aggregation by instead distributing algorithms to the data before centrally updating one global model. OBJECTIVE:To establish a multicenter collaboration and assess the feasibility of using FL to train machine learning models for intracranial hemorrhage (ICH) detection without sharing data between sites. METHODS:Five neurosurgery departments across the United States collaborated to establish a federated network and train a convolutional neural network to detect ICH on computed tomography scans. The global FL model was benchmarked against a standard, centrally trained model using a held-out data set and was compared against locally trained models using site data. RESULTS:A federated network of practicing neurosurgeon scientists was successfully initiated to train a model for predicting ICH. The FL model achieved an area under the ROC curve of 0.9487 (95% CI 0.9471-0.9503) when predicting all subtypes of ICH compared with a benchmark (non-FL) area under the ROC curve of 0.9753 (95% CI 0.9742-0.9764), although performance varied by subtype. The FL model consistently achieved top three performance when validated on any site's data, suggesting improved generalizability. A qualitative survey described the experience of participants in the federated network. CONCLUSION/CONCLUSIONS:This study demonstrates the feasibility of implementing a federated network for multi-institutional collaboration among clinicians and using FL to conduct machine learning research, thereby opening a new paradigm for neurosurgical collaboration.