Faculty Bibliography

OBJECTIVE/UNASSIGNED:This study investigates speech decoding from neural signals captured by intracranial electrodes. Most prior works can only work with electrodes on a 2D grid (i.e., Electrocorticographic or ECoG array) and data from a single patient. We aim to design a deep-learning model architecture that can accommodate both surface (ECoG) and depth (stereotactic EEG or sEEG) electrodes. The architecture should allow training on data from multiple participants with large variability in electrode placements and the trained model should perform well on participants unseen during training. APPROACH/UNASSIGNED:We propose a novel transformer-based model architecture named SwinTW that can work with arbitrarily positioned electrodes, by leveraging their 3D locations on the cortex rather than their positions on a 2D grid. We train both subject-specific models using data from a single participant as well as multi-patient models exploiting data from multiple participants. MAIN RESULTS/UNASSIGNED:The subject-specific models using only low-density 8x8 ECoG data achieved high decoding Pearson Correlation Coefficient with ground truth spectrogram (PCC=0.817), over N=43 participants, outperforming our prior convolutional ResNet model and the 3D Swin transformer model. Incorporating additional strip, depth, and grid electrodes available in each participant (N=39) led to further improvement (PCC=0.838). For participants with only sEEG electrodes (N=9), subject-specific models still enjoy comparable performance with an average PCC=0.798. The multi-subject models achieved high performance on unseen participants, with an average PCC=0.765 in leave-one-out cross-validation. SIGNIFICANCE/UNASSIGNED:The proposed SwinTW decoder enables future speech neuroprostheses to utilize any electrode placement that is clinically optimal or feasible for a particular participant, including using only depth electrodes, which are more routinely implanted in chronic neurosurgical procedures. Importantly, the generalizability of the multi-patient models suggests the exciting possibility of developing speech neuroprostheses for people with speech disability without relying on their own neural data for training, which is not always feasible.

Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.

OBJECTIVE:We retrospectively explored patients with drug-resistant epilepsy (DRE) who previously underwent presurgical evaluation to identify correlations between surgical outcomes and pathogenic variants in epilepsy genes. METHODS:Through an international collaboration, we evaluated adult DRE patients who were screened for surgical candidacy. Patients with pathogenic (P) or likely pathogenic (LP) germline variants in genes relevant to their epilepsy were included, regardless of whether the genetic diagnosis was made before or after the presurgical evaluation. Patients were divided into two groups: resective surgery (RS) and non-resective surgery candidates (NRSC), with the latter group further divided into: palliative surgery (vagus nerve stimulation, deep brain stimulation, responsive neurostimulation or corpus callosotomy) and no surgery. We compared surgical candidacy evaluations and postsurgical outcomes in patients with different genetic abnormalities. RESULTS:We identified 142 patients with P/LP variants. After presurgical evaluation, 36 patients underwent RS, while 106 patients were NRSC. Patients with variants in ion channel and synaptic transmission genes were more common in the NRSC group (48 %), compared with the RS group (14 %) (p<0.001). Most patients in the RS group had tuberous sclerosis complex. Almost half (17/36, 47 %) in the RS group had Engel class I or II outcomes. Patients with channelopathies were less likely to undergo a surgical procedure than patients with mTORopathies, but when deemed suitable for resection had better surgical outcomes (71 % versus 41 % with Engel I/II). Within the NRSC group, 40 underwent palliative surgery, with 26/40 (65 %) having ≥50 % seizure reduction after mean follow-up of 11 years. Favourable palliative surgery outcomes were observed across a diverse range of genetic epilepsies. SIGNIFICANCE/CONCLUSIONS:Genomic findings, including a channelopathy diagnosis, should not preclude presurgical evaluation or epilepsy surgery, and appropriately selected cases may have good surgical outcomes. Prospective registries of patients with monogenic epilepsies who undergo epilepsy surgery can provide additional insights on outcomes.

OBJECTIVE:New-onset refractory status epilepticus (NORSE) is a rare but severe clinical syndrome. Despite rigorous evaluation, the underlying cause is unknown in 30%-50% of patients and treatment strategies are largely empirical. The aim of this study was to describe clinical outcomes in a cohort of well-phenotyped, thoroughly investigated patients who survived the initial phase of cryptogenic NORSE managed in specialist centers. METHODS:Well-characterized cases of cryptogenic NORSE were identified through the EPIGEN and Critical Care EEG Monitoring Research Consortia (CCEMRC) during the period 2005-2019. Treating epileptologists reported on post-NORSE survival rates and sequelae in patients after discharge from hospital. Among survivors >6 months post-discharge, we report the rates and severity of active epilepsy, global disability, vocational, and global cognitive and mental health outcomes. We attempt to identify determinants of outcome. RESULTS:Among 48 patients who survived the acute phase of NORSE to the point of discharge from hospital, 9 had died at last follow-up, of whom 7 died within 6 months of discharge from the tertiary care center. The remaining 39 patients had high rates of active epilepsy as well as vocational, cognitive, and psychiatric comorbidities. The epilepsy was usually multifocal and typically drug resistant. Only a minority of patients had a good functional outcome. Therapeutic interventions were heterogenous during the acute phase of the illness. There was no clear relationship between the nature of treatment and clinical outcomes. SIGNIFICANCE/CONCLUSIONS:Among survivors of cryptogenic NORSE, longer-term outcomes in most patients were life altering and often catastrophic. Treatment remains empirical and variable. There is a pressing need to understand the etiology of cryptogenic NORSE and to develop tailored treatment strategies.

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.

OBJECTIVE:This study was undertaken to conduct external validation of previously published epilepsy surgery prediction tools using a large independent multicenter dataset and to assess whether these tools can stratify patients for being operated on and for becoming free of disabling seizures (International League Against Epilepsy stage 1 and 2). METHODS:We analyzed a dataset of 1562 patients, not used for tool development. We applied two scales: Epilepsy Surgery Grading Scale (ESGS) and Seizure Freedom Score (SFS); and two versions of Epilepsy Surgery Nomogram (ESN): the original version and the modified version, which included electroencephalographic data. For the ESNs, we used calibration curves and concordance indexes. We stratified the patients into three tiers for assessing the chances of attaining freedom from disabling seizures after surgery: high (ESGS = 1, SFS = 3-4, ESNs > 70%), moderate (ESGS = 2, SFS = 2, ESNs = 40%-70%), and low (ESGS = 2, SFS = 0-1, ESNs < 40%). We compared the three tiers as stratified by these tools, concerning the proportion of patients who were operated on, and for the proportion of patients who became free of disabling seizures. RESULTS:The concordance indexes for the various versions of the nomograms were between .56 and .69. Both scales (ESGS, SFS) and nomograms accurately stratified the patients for becoming free of disabling seizures, with significant differences among the three tiers (p < .05). In addition, ESGS and the modified ESN accurately stratified the patients for having been offered surgery, with significant difference among the three tiers (p < .05). SIGNIFICANCE/CONCLUSIONS:ESGS and the modified ESN (at thresholds of 40% and 70%) stratify patients undergoing presurgical evaluation into three tiers, with high, moderate, and low chance for favorable outcome, with significant differences between the groups concerning having surgery and becoming free of disabling seizures. Stratifying patients for epilepsy surgery has the potential to help select the optimal candidates in underprivileged areas and better allocate resources in developed countries.

Convolutional neural networks (CNN) show great promise for translating decades of research on structural abnormalities in temporal lobe epilepsy into clinical practice. Three-dimensional CNNs typically outperform two-dimensional CNNs in medical imaging. Here we explore for the first time whether a three-dimensional CNN outperforms a two-dimensional CNN for identifying temporal lobe epilepsy-specific features on MRI. Using 1178 T1-weighted images (589 temporal lobe epilepsy, 589 healthy controls) from 12 surgical centres, we trained 3D and 2D CNNs for temporal lobe epilepsy versus healthy control classification, using feature visualization to identify important regions. The 3D CNN was compared to the 2D model and to a randomized model (comparison to chance). Further, we explored the effect of sample size with subsampling, examined model performance based on single-subject clinical characteristics, and tested the impact of image harmonization on model performance. Across 50 datapoints (10 runs with 5-folds each) the 3D CNN median accuracy was 86.4% (35.3% above chance) and the median F1-score was 86.1% (33.3% above chance). The 3D model yielded higher accuracy compared to the 2D model on 84% of datapoints (median 2D accuracy, 83.0%), a significant outperformance for the 3D model (binomial test: P < 0.001). This advantage of the 3D model was only apparent at the highest sample size. Saliency maps exhibited the importance of medial-ventral temporal, cerebellar, and midline subcortical regions across both models for classification. However, the 3D model had higher salience in the most important regions, the ventral-medial temporal and midline subcortical regions. Importantly, the model achieved high accuracy (82% accuracy) even in patients without MRI-identifiable hippocampal sclerosis. Finally, applying ComBat for harmonization did not improve performance. These findings highlight the value of 3D CNNs for identifying subtle structural abnormalities on MRI, especially in patients without clinically identified temporal lobe epilepsy lesions. Our findings also reveal that the advantage of 3D CNNs relies on large sample sizes for model training.

Cortical regions supporting speech production are commonly established using neuroimaging techniques in both research and clinical settings. However, for neurosurgical purposes, structural function is routinely mapped peri-operatively using direct electrocortical stimulation. While this method is the gold standard for identification of eloquent cortical regions to preserve in neurosurgical patients, there is lack of specificity of the actual underlying cognitive processes being interrupted. To address this, we propose mapping the temporal dynamics of speech arrest across peri-sylvian cortices by quantifying the latency between stimulation and speech deficits. In doing so, we are able to substantiate hypotheses about distinct region-specific functional roles (e.g. planning versus motor execution). In this retrospective observational study, we analysed 20 patients (12 female; age range 14-43) with refractory epilepsy who underwent continuous extra-operative intracranial EEG monitoring of an automatic speech task during clinical bedside language mapping. Latency to speech arrest was calculated as time from stimulation onset to speech arrest onset, controlling for individual speech rate. Most instances of motor-based arrest (87.5% of 96 instances) were in sensorimotor cortex with mid-range latencies to speech arrest with a distributional peak at 0.47 s. Speech arrest occurred in numerous regions, with relatively short latencies in supramarginal gyrus (0.46 s), superior temporal gyrus (0.51 s) and middle temporal gyrus (0.54 s), followed by relatively long latencies in sensorimotor cortex (0.72 s) and especially long latencies in inferior frontal gyrus (0.95 s). Non-parametric testing for speech arrest revealed that region predicted latency; latencies in supramarginal gyrus and in superior temporal gyrus were shorter than in sensorimotor cortex and in inferior frontal gyrus. Sensorimotor cortex is primarily responsible for motor-based arrest. Latencies to speech arrest in supramarginal gyrus and superior temporal gyrus (and to a lesser extent middle temporal gyrus) align with latencies to motor-based arrest in sensorimotor cortex. This pattern of relatively quick cessation of speech suggests that stimulating these regions interferes with the outgoing motor execution. In contrast, the latencies to speech arrest in inferior frontal gyrus and in ventral regions of sensorimotor cortex were significantly longer than those in temporoparietal regions. Longer latencies in the more frontal areas (including inferior frontal gyrus and ventral areas of precentral gyrus and postcentral gyrus) suggest that stimulating these areas interrupts a higher-level speech production process involved in planning. These results implicate the ventral specialization of sensorimotor cortex (including both precentral and postcentral gyri) for speech planning above and beyond motor execution.

PURPOSE/OBJECTIVE:Brain responsive neurostimulation (NeuroPace) treats patients with refractory focal epilepsy and provides chronic electrocorticography (ECoG). We explored how machine learning algorithms applied to interictal ECoG could assess clinical response to changes in neurostimulation parameters. METHODS:We identified five responsive neurostimulation patients each with ≥200 continuous days of stable medication and detection settings (median, 358 days per patient). For each patient, interictal ECoG segments for each month were labeled as "high" or "low" to represent relatively high or low long-episode (i.e., seizure) count compared with the median monthly long-episode count. Power from six conventional frequency bands from four responsive neurostimulation channels were extracted as features. For each patient, five machine learning algorithms were trained on 80% of ECoG, then tested on the remaining 20%. Classifiers were scored by the area-under-the-receiver-operating-characteristic curve. We explored how individual circadian cycles of seizure activity could inform classifier building. RESULTS:Support vector machine or gradient boosting models achieved the best performance, ranging from 0.705 (fair) to 0.892 (excellent) across patients. High gamma power was the most important feature, tending to decrease during low-seizure-frequency epochs. For two subjects, training on ECoG recorded during the circadian ictal peak resulted in comparable model performance, despite less data used. CONCLUSIONS:Machine learning analysis on retrospective background ECoG can classify relative seizure frequency for an individual patient. High gamma power was the most informative, whereas individual circadian patterns of seizure activity can guide model building. Machine learning classifiers built on interictal ECoG may guide stimulation programming.