Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm
Triola, Marc M; Reinstein, Ilan; Marin, Marina; Gillespie, Colleen; Abramson, Steven; Grossman, Robert I; Rivera, Rafael
PURPOSE/OBJECTIVE:To explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. METHOD/METHODS:Using application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. RESULTS:The retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). CONCLUSIONS:The virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.
Signatures of medical student applicants and academic success
Baron, Tal; Grossman, Robert I; Abramson, Steven B; Pusic, Martin V; Rivera, Rafael; Triola, Marc M; Yanai, Itai
The acceptance of students to a medical school places a considerable emphasis on performance in standardized tests and undergraduate grade point average (uGPA). Traditionally, applicants may be judged as a homogeneous population according to simple quantitative thresholds that implicitly assume a linear relationship between scores and academic success. This 'one-size-fits-all' approach ignores the notion that individuals may show distinct patterns of achievement and follow diverse paths to success. In this study, we examined a dataset composed of 53 variables extracted from the admissions application records of 1,088 students matriculating to NYU School of Medicine between the years 2006-2014. We defined training and test groups and applied K-means clustering to search for distinct groups of applicants. Building an optimized logistic regression model, we then tested the predictive value of this clustering for estimating the success of applicants in medical school, aggregating eight performance measures during the subsequent medical school training as a success factor. We found evidence for four distinct clusters of students-we termed 'signatures'-which differ most substantially according to the absolute level of the applicant's uGPA and its trajectory over the course of undergraduate education. The 'risers' signature showed a relatively higher uGPA and also steeper trajectory; the other signatures showed each remaining combination of these two main factors: 'improvers' relatively lower uGPA, steeper trajectory; 'solids' higher uGPA, flatter trajectory; 'statics' both lower uGPA and flatter trajectory. Examining the success index across signatures, we found that the risers and the statics have significantly higher and lower likelihood of quantifiable success in medical school, respectively. We also found that each signature has a unique set of features that correlate with its success in medical school. The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.
Grossman, Robert I
Quantification of normal-appearing white matter tract integrity in multiple sclerosis: a diffusion kurtosis imaging study
de Kouchkovsky, Ivan; Fieremans, Els; Fleysher, Lazar; Herbert, Joseph; Grossman, Robert I; Inglese, Matilde
Our aim was to characterize the nature and extent of pathological changes in the normal-appearing white matter (NAWM) of patients with multiple sclerosis (MS) using novel diffusion kurtosis imaging-derived white matter tract integrity (WMTI) metrics and to investigate the association between these WMTI metrics and clinical parameters. Thirty-two patients with relapsing-remitting MS and 19 age- and gender-matched healthy controls underwent MRI and neurological examination. Maps of mean diffusivity, fractional anisotropy and WMTI metrics (intra-axonal diffusivity, axonal water fraction, tortuosity and axial and radial extra-axonal diffusivity) were created. Tract-based spatial statistics analysis was performed to assess for differences in the NAWM between patients and controls. A region of interest analysis of the corpus callosum was also performed to assess for group differences and to evaluate correlations between WMTI metrics and measures of disease severity. Mean diffusivity and radial extra-axonal diffusivity were significantly increased while fractional anisotropy, axonal water fraction, intra-axonal diffusivity and tortuosity were decreased in MS patients compared with controls (p values ranging from <0.001 to <0.05). Axonal water fraction in the corpus callosum was significantly associated with the expanded disability status scale score (rho = -0.39, p = 0.035). With the exception of the axial extra-axonal diffusivity, all metrics were correlated with the symbol digits modality test score (p values ranging from 0.001 to <0.05). WMTI metrics are thus sensitive to changes in the NAWM of MS patients and might provide a more pathologically specific, clinically meaningful and practical complement to standard diffusion tensor imaging-derived metrics.
MR Imaging Applications in Mild Traumatic Brain Injury: An Imaging Update
Wu, Xin; Kirov, Ivan I; Gonen, Oded; Ge, Yulin; Grossman, Robert I; Lui, Yvonne W
Mild traumatic brain injury (mTBI), also commonly referred to as concussion, affects millions of Americans annually. Although computed tomography is the first-line imaging technique for all traumatic brain injury, it is incapable of providing long-term prognostic information in mTBI. In the past decade, the amount of research related to magnetic resonance (MR) imaging of mTBI has grown exponentially, partly due to development of novel analytical methods, which are applied to a variety of MR techniques. Here, evidence of subtle brain changes in mTBI as revealed by these techniques, which are not demonstrable by conventional imaging, will be reviewed. These changes can be considered in three main categories of brain structure, function, and metabolism. Macrostructural and microstructural changes have been revealed with three-dimensional MR imaging, susceptibility-weighted imaging, diffusion-weighted imaging, and higher order diffusion imaging. Functional abnormalities have been described with both task-mediated and resting-state blood oxygen level-dependent functional MR imaging. Metabolic changes suggesting neuronal injury have been demonstrated with MR spectroscopy. These findings improve understanding of the true impact of mTBI and its pathogenesis. Further investigation may eventually lead to improved diagnosis, prognosis, and management of this common and costly condition. ((c)) RSNA, 2016.
N-acetyl-aspartate levels correlate with intra-axonal compartment parameters from diffusion MRI
Grossman, Elan J; Kirov, Ivan I; Gonen, Oded; Novikov, Dmitry S; Davitz, Matthew S; Lui, Yvonne W; Grossman, Robert I; Inglese, Matilde; Fieremans, Els
Diffusion MRI combined with biophysical modeling allows for the description of a white matter (WM) fiber bundle in terms of compartment specific white matter tract integrity (WMTI) metrics, which include intra-axonal diffusivity (Daxon), extra-axonal axial diffusivity (De||), extra-axonal radial diffusivity (De upper left and right quadrants), axonal water fraction (AWF), and tortuosity (alpha) of extra-axonal space. Here we derive these parameters from diffusion kurtosis imaging to examine their relationship to concentrations of global WM N-acetyl-aspartate (NAA), creatine (Cr), choline (Cho) and myo-Inositol (mI), as measured with proton MR spectroscopy (1H-MRS), in a cohort of 25 patients with mild traumatic brain injury (MTBI). We found statistically significant (p<0.05) positive correlations between NAA and Daxon, AWF, alpha, and fractional anisotropy; negative correlations between NAA and De, upper left and right quadrants and the overall radial diffusivity (D upper left and right quadrants). These correlations were supported by similar findings in regional analysis of the genu and splenium of the corpus callosum. Furthermore, a positive correlation in global WM was noted between Daxon and Cr, as well as a positive correlation between De|| and Cho, and a positive trend between De|| and mI. The specific correlations between NAA, an endogenous probe of the neuronal intracellular space, and WMTI metrics related to the intra-axonal space, combined with the specific correlations of De|| with mI and Cho, both predominantly present extra-axonally, corroborate the overarching assumption of many advanced modeling approaches that diffusion imaging can disentangle between the intra- and extra-axonal compartments in WM fiber bundles. Our findings are also generally consistent with what is known about the pathophysiology of MTBI, which appears to involve both intra-axonal injury (as reflected by a positive trend between NAA and Daxon) as well as axonal shrinkage, demyelination, degeneration, and/or loss (as reflected by correlations between NAA and De upper left and right quadrants, AWF, and alpha).
Non-Gaussian diffusion MRI of gray matter is associated with cognitive impairment in multiple sclerosis
Bester, M; Jensen, J H; Babb, J S; Tabesh, A; Miles, L; Herbert, J; Grossman, R I; Inglese, M
BACKGROUND: Non-Gaussian diffusion imaging by using diffusional kurtosis imaging (DKI) allows assessment of isotropic tissue as of gray matter (GM), an important limitation of diffusion tensor imaging (DTI). OBJECTIVE: In this study, we describe DKI and DTI metrics of GM in multiple sclerosis (MS) patients and their association with cognitive deficits. METHODS: Thirty-four patients with relapsing-remitting MS and 17 controls underwent MRI on a 3T scanner including a sequence for DKI with 30 diffusion directions and 3b values for each direction. Mean kurtosis (MK), mean diffusivity and fractional anisotropy (FA) of cortical and subcortical GM were measured using histogram analysis. Spearman rank correlations were used to characterize associations among imaging measures and clinical/neuropsychological scores. RESULTS: In cortical GM, a significant decrease of MK (0.68 vs. 0.73; p < 0.001) and increase of FA (0.16 vs. 0.13; p < 0.001) was found in patients compared to controls. Decreased cortical MK was correlated with poor performance on the Delis-Kaplan Executive Function System test (r = 0.66, p = 0.01). CONCLUSION: Mean kurtosis is sensitive to abnormality in GM of MS patients and can provide information that is complementary to that of conventional DTI-derived metrics. The association between MK and cognitive deficits suggests that DKI might serve as a clinically relevant biomarker for cortical injury.
Relationship between iron accumulation and white matter injury in multiple sclerosis: a case-control study
Raz, Eytan; Branson, Brittany; Jensen, Jens H; Bester, Maxim; Babb, James S; Herbert, Joseph; Grossman, Robert I; Inglese, Matilde
Despite the increasing development and applications of iron imaging, the pathophysiology of iron accumulation in multiple sclerosis (MS), and its role in disease progression and development of clinical disability, is poorly understood. The aims of our study were to determine the presence and extent of iron in T2 visible lesions and gray and white matter using magnetic field correlation (MFC) MRI and correlate with microscopic white matter (WM) injury as measured by diffusion tensor imaging (DTI). This is a case-control study including a series of 31 patients with clinically definite MS. The mean age was 39 years [standard deviation (SD) = 9.55], they were 11 males and 20 females, with a disease duration average of 3 years (range 0-13) and a median EDSS of 2 (0-4.5). Seventeen healthy volunteers (6 males and 11 females) with a mean age of 36 years (SD = 11.4) were recruited. All subjects underwent MR imaging on a 3T scanner using T2-weighted sequence, 3D T1 MPRAGE, MFC, single-shot DTI and post-contrast T1. T2-lesion volumes, brain volumetry, DTI parameters and iron quantification were calculated and multiple correlations were exploited. Increased MFC was found in the putamen (p = 0.061), the thalamus (p = 0.123), the centrum semiovale (p = 0.053), globus pallidus (p = 0.008) and gray matter (GM) (p = 0.004) of MS patients compared to controls. The mean lesional MFC was 121 s-2 (SD = 67), significantly lower compared to the GM MFC (<0.0001). The GM mean diffusivity (MD) was inversely correlated with the MFC in the centrum semiovale (p < 0.001), and in the splenium of the corpus callosum (p < 0.001). Patients with MS have increased iron in the globus pallidus, putamen and centrum with a trend toward increased iron in all the brain structures. Quantitative iron evaluation of WM and GM may improve the understanding of MS pathophysiology, and might serve as a surrogate marker of disease progression.
Gray Matter Correlates of Cognitive Performance Differ between Relapsing-Remitting and Primary-Progressive Multiple Sclerosis
Jonkman, Laura E; Rosenthal, Diana M; Sormani, Maria Pia; Miles, Laura; Herbert, Joseph; Grossman, Robert I; Inglese, Matilde
Multiple Sclerosis (MS) is a chronic inflammatory/demyelinating and neurodegenerative disease of the central nervous system (CNS). Most patients experience a relapsing-remitting (RR) course, while about 15-20% of patients experience a primary progressive (PP) course. Cognitive impairment affects approximately 40-70% of all MS patients and differences in cognitive impairment between RR-MS and PP-MS have been found. We aimed to compare RR-MS and PP-MS patients in terms of cognitive performance, and to investigate the MRI correlates of cognitive impairment in the two groups using measures of brain volumes and cortical thickness. Fifty-seven patients (42 RR-MS, 15 PP-MS) and thirty-eight matched controls underwent neuropsychological (NP) testing and MRI. PP-MS patients scored lower than RR-MS patients on most of the NP tests in absence of any specific pattern. PP-MS patients showed significantly lower caudate volume. There was no significant difference in MRI correlates of cognitive impairment between the two groups except for a prevalent association with MRI measures of cortical GM injury in RR-MS patients and with MRI measures of subcortical GM injury in PP-MS patients. This suggests that although cognitive impairment results from several factors, cortical and subcortical GM injury may play a different role depending on the disease course.
Disrupted blood flow modulation in functional brain networks in multiple sclerosis measured with hypercapnia MRI [Meeting Abstract]
Ge, Y; Marshall, O; Pape, L; Lu, H; Kister, I; Grossman, RI