Faculty Bibliography

Naturalistic electrocorticography (ECoG) data are a rare but essential resource for studying the brain's linguistic capabilities. ECoG offers high temporal resolution suitable for investigating processes at multiple temporal timescales and frequency bands. It also provides broad spatial coverage, often along critical language areas. Here, we share a dataset of nine ECoG participants with 1,330 electrodes listening to a 30-minute audio podcast. The richness of this naturalistic stimulus can be used for various research questions, from auditory perception to narrative integration. In addition to the neural data, we extracted linguistic features of the stimulus ranging from phonetic information to large language model word embeddings. We use these linguistic features in encoding models that relate stimulus properties to neural activity. Finally, we provide detailed tutorials for preprocessing raw data, extracting stimulus features, and running encoding analyses that can serve as a pedagogical resource or a springboard for new research.

Neuronal oscillations at about 10 Hz, called alpha oscillations, are often thought to arise from synchronous activity across the occipital cortex and are usually largest when the cortex is inactive. However, recent studies measuring visual receptive fields have reported that local alpha power increases when cortex is excited by visual stimulation. This contrasts with the expectation that alpha oscillations are associated with cortical inactivity. Here, we used intracranial electrodes in human patients to measure alpha oscillations in response to visual stimuli whose location varied systematically across the visual field. We hypothesized that stimulus-driven local increases in alpha power result from a mixture of two effects: a reduction in alpha oscillatory power and a simultaneous increase in broadband power. To test this, we implemented a model to separate these components. The two components were then independently fit by population receptive field (pRF) models. We find that the alpha pRFs have similar center locations to pRFs estimated from broadband power but are several times larger and exhibit the opposite effect: alpha oscillatory power decreases in response to stimuli within the receptive field, reinforcing the link between alpha oscillations and cortical inactivity, whereas broadband power increases. The results demonstrate that alpha suppression in the human visual cortex can be precisely tuned, but that to measure these effects, it is essential to separate the oscillatory signal from broadband power changes. Finally, we show how the large size and the negative valence of alpha pRFs can explain key features of exogenous visual attention.

Despite decades of advancements in diagnostic MRI, 30-50% of temporal lobe epilepsy (TLE) patients remain categorized as "non-lesional" (i.e., MRI negative or MRI-) based on visual assessment by human experts. MRI- patients face diagnostic uncertainty and significant delays in treatment planning. Quantitative MRI studies have demonstrated that MRI- patients often exhibit a TLE-specific pattern of temporal and limbic atrophy that may be too subtle for the human eye to detect. This signature pattern could be successfully translated into clinical use via artificial intelligence (AI) advances in computer-aided MRI interpretation, thereby improving the detection of brain "lesional" patterns associated with TLE. Here, we tested this hypothesis by employing a three-dimensional convolutional neural network (3D CNN) applied to a dataset of 1,178 scans from 12 different centers. 3D CNN was able to differentiate TLE from healthy controls with high accuracy (85.9% ± 2.8), significantly outperforming support vector machines based on hippocampal (74.4% ± 2.6) and whole-brain (78.3% ± 3.3) volumes. Our analysis subsequently focused on a subset of patients who achieved sustained seizure freedom post-surgery as a gold standard for confirming TLE. Importantly, MRI- patients from this cohort were accurately identified as TLE 82.7% ± 0.9 of the time, an encouraging finding since clinically these were all patients considered to be MRI- (i.e., not radiographically different than controls). The saliency maps from the CNN revealed that limbic structures, particularly medial temporal, cingulate, and orbitofrontal areas, were most influential in classification, confirming the importance of the well-established TLE signature atrophy pattern for diagnosis. Indeed, the saliency maps were similar in MRI+ and MRI- TLE groups, suggesting that even when humans cannot distinguish more subtle levels of atrophy, these MRI- patients are on the same continuum common across all TLE patients. As such, AI can identify TLE lesional patterns and AI-aided diagnosis has the potential to greatly enhance the neuroimaging diagnosis of TLE and redefine the concept of "lesional" TLE.

Sentence production is the uniquely human ability to transform complex thoughts into strings of words. Despite the importance of this process, language production research has primarily focused on single words. It remains a largely untested assumption that the principles of word production generalize to more naturalistic utterances like sentences. Here, we investigate this using high-resolution neurosurgical recordings (ECoG) and an overt production experiment where ten patients produced six words in isolation (picture naming) and in sentences (scene description). We trained machine learning classifiers to identify the unique brain activity patterns for each word during picture naming, and used these patterns to decode which words patients were processing while they produced sentences. Our findings confirm that words share cortical representations across tasks, but reveal a division of labor within the language network. In sensorimotor cortex, words were consistently activated in the order in which they were said in the sentence. However, in prefrontal cortex, the order in which words were processed depended on the syntactic structure of the sentence. In non-canonical sentences (passives), we further observed a spatial code for syntactic roles, with subjects selectively encoded in inferior frontal gyrus (IFG) and objects selectively encoded in middle frontal gyrus (MFG). We suggest that these complex dynamics of prefrontal cortex may impose a subtle pressure on language evolution, potentially explaining why nearly all the world's languages position subjects before objects.

We introduce an intracranial EEG (iEEG) dataset collected as part of an adversarial collaboration between proponents of two theories of consciousness: Global Neuronal Workspace Theory and Integrated Information Theory. The data were recorded from 38 patients undergoing intracranial monitoring of epileptic seizures across three research centers using the same experimental protocol. Participants were presented with suprathreshold visual stimuli belonging to four different categories (faces, objects, letters, false fonts) in three orientations (front, left, right view), and for three durations (0.5, 1.0, 1.5 s). Participants engaged in a non-speeded Go/No-Go target detection task to identify infrequent targets with some stimuli becoming task-relevant and others task-irrelevant. Participants also engaged in a motor localizer task. The data were checked for its quality and converted to Brain Imaging Data Structure (BIDS). The de-identified dataset contains demographics, clinical information, electrode reconstruction, behavioral performance, and eye-tracking data. We also provide code to preprocess and analyze the data. This dataset holds promise for reuse in consciousness science and vision neuroscience to answer questions related to stimulus processing, target detection, and task-relevance, among many others.

The ability to connect the form and meaning of a concept, known as word retrieval, is fundamental to human communication. While various input modalities could lead to identical word retrieval, the exact neural dynamics supporting this process relevant to daily auditory discourse remain poorly understood. Here, we recorded neurosurgical electrocorticography (ECoG) data from 48 patients and dissociated two key language networks that highly overlap in time and space, critical for word retrieval. Using unsupervised temporal clustering techniques, we found a semantic processing network located in the middle and inferior frontal gyri. This network was distinct from an articulatory planning network in the inferior frontal and precentral gyri, which was invariant to input modalities. Functionally, we confirmed that the semantic processing network encodes word surprisal during sentence perception. These findings elucidate neurophysiological mechanisms underlying the processing of semantic auditory inputs ranging from passive language comprehension to conversational speech.

This study introduces a unified computational framework connecting acoustic, speech and word-level linguistic structures to study the neural basis of everyday conversations in the human brain. We used electrocorticography to record neural signals across 100 h of speech production and comprehension as participants engaged in open-ended real-life conversations. We extracted low-level acoustic, mid-level speech and contextual word embeddings from a multimodal speech-to-text model (Whisper). We developed encoding models that linearly map these embeddings onto brain activity during speech production and comprehension. Remarkably, this model accurately predicts neural activity at each level of the language processing hierarchy across hours of new conversations not used in training the model. The internal processing hierarchy in the model is aligned with the cortical hierarchy for speech and language processing, where sensory and motor regions better align with the model's speech embeddings, and higher-level language areas better align with the model's language embeddings. The Whisper model captures the temporal sequence of language-to-speech encoding before word articulation (speech production) and speech-to-language encoding post articulation (speech comprehension). The embeddings learned by this model outperform symbolic models in capturing neural activity supporting natural speech and language. These findings support a paradigm shift towards unified computational models that capture the entire processing hierarchy for speech comprehension and production in real-world conversations.

Syntax, the abstract structure of language, is a hallmark of human cognition. Despite its importance, its neural underpinnings remain obscured by inherent limitations of non-invasive brain measures and a near total focus on comprehension paradigms. Here, we address these limitations with high-resolution neurosurgical recordings (electrocorticography) and a controlled sentence production experiment. We uncover three syntactic networks that are broadly distributed across traditional language regions, but with focal concentrations in middle and inferior frontal gyri. In contrast to previous findings from comprehension studies, these networks process syntax mostly to the exclusion of words and meaning, supporting a cognitive architecture with a distinct syntactic system. Most strikingly, our data reveal an unexpected property of syntax: it is encoded independent of neural activity levels. We propose that this "low-activity coding" scheme represents a novel mechanism for encoding information, reserved for higher-order cognition more broadly.

The ability to connect the form and meaning of a concept, known as word retrieval, is fundamental to human communication. While various input modalities could lead to identical word retrieval, the exact neural dynamics supporting this convergence relevant to daily auditory discourse remain poorly understood. Here, we leveraged neurosurgical electrocorticographic (ECoG) recordings from 48 patients and dissociated two key language networks that highly overlap in time and space integral to word retrieval. Using unsupervised temporal clustering techniques, we found a semantic processing network located in the middle and inferior frontal gyri. This network was distinct from an articulatory planning network in the inferior frontal and precentral gyri, which was agnostic to input modalities. Functionally, we confirmed that the semantic processing network encodes word surprisal during sentence perception. Our findings characterize how humans integrate ongoing auditory semantic information over time, a critical linguistic function from passive comprehension to daily discourse.

Resective surgery for drug-resistant temporal lobe epilepsy remains underutilized in the United States. While anteromesial temporal lobectomy consistently achieves the highest rates of long-term seizure freedom, it comes with greater risks for memory and language decline. Magnetic resonance imaging-guided laser interstitial thermal therapy and neuromodulation have gained popularity due to perceived lower surgical risk and faster recovery, although they yield lower rates of sustained seizure freedom. Neuromodulation with vagus nerve, deep brain, or responsive neurostimulation provides an option for patients ineligible for resection or ablation, but overall seizure outcomes remain modest. Balancing improved seizure control with open resection against the potential cognitive advantages of less invasive treatments is complex, requiring careful patient selection. Future research must refine these approaches to optimize results. Thoughtful, individualized decision-making, guided by each patient's clinical scenario and goals, is paramount for achieving the best balance between seizure freedom, cognitive preservation, and overall patient outcome.