Faculty Bibliography

The most critical attribute of human language is its unbounded combinatorial nature: smaller elements can be combined into larger structures on the basis of a grammatical system, resulting in a hierarchy of linguistic units, such as words, phrases and sentences. Mentally parsing and representing such structures, however, poses challenges for speech comprehension. In speech, hierarchical linguistic structures do not have boundaries that are clearly defined by acoustic cues and must therefore be internally and incrementally constructed during comprehension. We found that, during listening to connected speech, cortical activity of different timescales concurrently tracked the time course of abstract linguistic structures at different hierarchical levels, such as words, phrases and sentences. Notably, the neural tracking of hierarchical linguistic structures was dissociated from the encoding of acoustic cues and from the predictability of incoming words. Our results indicate that a hierarchy of neural processing timescales underlies grammar-based internal construction of hierarchical linguistic structure.

Brain activity can follow the rhythms of dynamic sensory stimuli, such as speech and music, a phenomenon called neural entrainment. It has been hypothesized that low-frequency neural entrainment in the neural delta and theta bands provides a potential mechanism to represent and integrate temporal information. Low-frequency neural entrainment is often studied using periodically changing stimuli and is analyzed in the frequency domain using the Fourier analysis. The Fourier analysis decomposes a periodic signal into harmonically related sinusoids. However, it is not intuitive how these harmonically related components are related to the response waveform. Here, we explain the interpretation of response harmonics, with a special focus on very low-frequency neural entrainment near 1 Hz. It is illustrated why neural responses repeating at f Hz do not necessarily generate any neural response at f Hz in the Fourier spectrum. A strong neural response at f Hz indicates that the time scales of the neural response waveform within each cycle match the time scales of the stimulus rhythm. Therefore, neural entrainment at very low frequency implies not only that the neural response repeats at f Hz but also that each period of the neural response is a slow wave matching the time scale of a f Hz sinusoid.

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

Perception is an active inferential process in which prior knowledge is combined with sensory input, the result of which determines the contents of awareness. Accordingly, previous experience is known to help the brain "decide" what to perceive. However, a critical aspect that has not been addressed is that previous experience can exert 2 opposing effects on perception: An attractive effect, sensitizing the brain to perceive the same again (hysteresis), or a repulsive effect, making it more likely to perceive something else (adaptation). We used functional magnetic resonance imaging and modeling to elucidate how the brain entertains these 2 opposing processes, and what determines the direction of such experience-dependent perceptual effects. We found that although affecting our perception concurrently, hysteresis and adaptation map into distinct cortical networks: a widespread network of higher-order visual and fronto-parietal areas was involved in perceptual stabilization, while adaptation was confined to early visual areas. This areal and hierarchical segregation may explain how the brain maintains the balance between exploiting redundancies and staying sensitive to new information. We provide a Bayesian model that accounts for the coexistence of hysteresis and adaptation by separating their causes into 2 distinct terms: Hysteresis alters the prior, whereas adaptation changes the sensory evidence (the likelihood function).

Upon repetition, certain stimuli induce reduced neural responses (i.e., repetition suppression), whereas others evoke stronger signals (i.e., repetition enhancement). It has been hypothesized that stimulus properties (e.g., visibility) determine the direction of the repetition effect. Here, we show that the very same stimuli can induce both repetition suppression and enhancement, whereby the only determining factor is the number of repetitions. Repeating the same, initially novel low-visible pictures of scenes for up to 5 times enhanced the blood oxygen level-dependent (BOLD) response in scene-selective areas, that is, the parahippocampal place area (PPA) and the transverse occipital sulcus (TOS), presumably reflecting the strengthening of the internal representation. Additional repetitions (6-9) resulted in progressively attenuated neural responses indicating a more efficient representation of the now familiar stimulus. Behaviorally, repetition led to increasingly faster responses and higher visibility ratings. Novel scenes induced the largest BOLD response in the PPA and also higher activity in yet another scene-selective region, the retrospenial cortex (RSC). We propose that 2 separable processes modulate activity in the PPA: one process optimizes the internal stimulus representation and involves TOS and the other differentiates between familiar and novel scenes and involves RSC.

Whether an object captures our attention depends on its bottom-up salience, that is, how different it is compared with its neighbors, and top-down control, that is, our current inner goals. At which neuronal stage they interact to guide behavior is still unknown. In a functional magnetic resonance imaging study, we found evidence for a hierarchy of saliency maps in human early visual cortex (V1 to hV4) and identified where bottom-up saliency interacts with top-down control: V1 represented pure bottom-up signals, V2 was only responsive to top-down modulations, and in hV4 bottom-up saliency and top-down control converged. Two distinct cerebral networks exerted top-down control: distractor suppression engaged the left intraparietal sulcus, while target enhancement involved the frontal eye field and lateral occipital cortex. Hence, attentional selection is implemented in integrated maps in visual cortex, which provide precise topographic information about target-distractor locations thus allowing for successful visual search.

Which neural processes underlie our conscious experience? One theoretical view argues that the neural correlates of consciousness (NCC) reside in local activity in sensory cortices. Accordingly, local category-specific gamma band responses in visual cortex correlate with conscious perception. However, as most studies manipulated conscious perception by altering the amount of sensory evidence, it is possible that they reflect prerequisites or consequences of consciousness rather than the actual NCC. Here we directly address this issue by developing a new experimental paradigm in which conscious perception is modulated either by sensory evidence or by previous exposure of the images while recording intracranial EEG from the higher-order visual cortex of human epilepsy patients. A clear prediction is that neural processes directly reflecting conscious perception should be present regardless of how it comes about. In contrast, we observed that although subjective reports were modulated both by sensory evidence and by previous exposure, gamma band responses solely reflected sensory evidence. This result contradicts the proposal that local gamma band responses in the higher-order visual cortex reflect conscious perception.

Spoken sentence comprehension relies on rapid and effortless temporal integration of speech units displayed at different rates. Temporal integration refers to how chunks of information perceived at different time scales are linked together by the listener in mapping speech sounds onto meaning. The neural implementation of this integration remains unclear. This study explores the role of short and long windows of integration in accessing meaning from long samples of speech. In a cross-linguistic study, we explore the time course of oscillatory brain activity between 1 and 100 Hz, recorded using EEG, during the processing of native and foreign languages. We compare oscillatory responses in a group of Italian and Spanish native speakers while they attentively listen to Italian, Japanese, and Spanish utterances, played either forward or backward. The results show that both groups of participants display a significant increase in gamma band power (55-75 Hz) only when they listen to their native language played forward. The increase in gamma power starts around 1000 msec after the onset of the utterance and decreases by its end, resembling the time course of access to meaning during speech perception. In contrast, changes in low-frequency power show similar patterns for both native and foreign languages. We propose that gamma band power reflects a temporal binding phenomenon concerning the coordination of neural assemblies involved in accessing meaning of long samples of speech.

Solving the problem of consciousness remains one of the biggest challenges in modern science. One key step towards understanding consciousness is to empirically narrow down neural processes associated with the subjective experience of a particular content. To unravel these neural correlates of consciousness (NCC) a common scientific strategy is to compare perceptual conditions in which consciousness of a particular content is present with those in which it is absent, and to determine differences in measures of brain activity (the so called "contrastive analysis"). However, this comparison appears not to reveal exclusively the NCC, as the NCC proper can be confounded with prerequisites for and consequences of conscious processing of the particular content. This implies that previous results cannot be unequivocally interpreted as reflecting the neural correlates of conscious experience. Here we review evidence supporting this conjecture and suggest experimental strategies to untangle the NCC from the prerequisites and consequences of conscious experience in order to further develop the otherwise valid and valuable contrastive methodology.

During meditation, practitioners are required to center their attention on a specific object for extended periods of time. When their thoughts get diverted, they learn to quickly disengage from the distracter. We hypothesized that learning to respond to the dual demand of engaging attention on specific objects and disengaging quickly from distracters enhances the efficiency by which meditation practitioners can allocate attention. We tested this hypothesis in a global-to-local task while measuring electroencephalographic activity from a group of eight highly trained Buddhist monks and nuns and a group of eight age and education matched controls with no previous meditation experience. Specifically, we investigated the effect of attentional training on the global precedence effect, i.e., faster detection of targets on a global than on a local level. We expected to find a reduced global precedence effect in meditation practitioners but not in controls, reflecting that meditators can more quickly disengage their attention from the dominant global level. Analysis of reaction times confirmed this prediction. To investigate the underlying changes in brain activity and their time course, we analyzed event-related potentials. Meditators showed an enhanced ability to select the respective target level, as reflected by enhanced processing of target level information. In contrast with control group, which showed a local target selection effect only in the P1 and a global target selection effect in the P3 component, meditators showed effects of local information processing in the P1, N2, and P3 and of global processing for the N1, N2, and P3. Thus, meditators seem to display enhanced depth of processing. In addition, meditation altered the uptake of information such that meditators selected target level information earlier in the processing sequence than controls. In a longitudinal experiment, we could replicate the behavioral effects, suggesting that meditation modulates attention already after a 4-day meditation retreat. Together, these results suggest that practicing meditation enhances the speed with which attention can be allocated and relocated, thus increasing the depth of information processing and reducing response latency.