Faculty Bibliography

Perception of number and space are tightly intertwined. It has been proposed that this is due to 'cortical recycling', where numerosity processing takes over circuits originally processing space. Do such 'recycled' circuits retain their original functionality? Here, we investigate interactions between numerosity and motion direction, two functions that both localize to parietal cortex. We describe a new phenomenon in which visual motion direction adapts nonsymbolic numerosity perception, giving rise to a repulsive aftereffect: motion to the left adapts small numbers, leading to overestimation of numerosity, while motion to the right adapts large numbers, resulting in underestimation. The reference frame of this effect is spatiotopic. Together with the tuning properties of the effect this suggests that motion direction-numerosity cross-adaptation may occur in a homolog of area LIP. 'Cortical recycling' thus expands but does not obliterate the functions originally performed by the recycled circuit, allowing for shared computations across domains.

The most critical attribute of human language is its unbounded combinatorial nature: smaller elements can be combined into larger structures on the basis of a grammatical system, resulting in a hierarchy of linguistic units, such as words, phrases and sentences. Mentally parsing and representing such structures, however, poses challenges for speech comprehension. In speech, hierarchical linguistic structures do not have boundaries that are clearly defined by acoustic cues and must therefore be internally and incrementally constructed during comprehension. We found that, during listening to connected speech, cortical activity of different timescales concurrently tracked the time course of abstract linguistic structures at different hierarchical levels, such as words, phrases and sentences. Notably, the neural tracking of hierarchical linguistic structures was dissociated from the encoding of acoustic cues and from the predictability of incoming words. Our results indicate that a hierarchy of neural processing timescales underlies grammar-based internal construction of hierarchical linguistic structure.

Brain activity can follow the rhythms of dynamic sensory stimuli, such as speech and music, a phenomenon called neural entrainment. It has been hypothesized that low-frequency neural entrainment in the neural delta and theta bands provides a potential mechanism to represent and integrate temporal information. Low-frequency neural entrainment is often studied using periodically changing stimuli and is analyzed in the frequency domain using the Fourier analysis. The Fourier analysis decomposes a periodic signal into harmonically related sinusoids. However, it is not intuitive how these harmonically related components are related to the response waveform. Here, we explain the interpretation of response harmonics, with a special focus on very low-frequency neural entrainment near 1 Hz. It is illustrated why neural responses repeating at f Hz do not necessarily generate any neural response at f Hz in the Fourier spectrum. A strong neural response at f Hz indicates that the time scales of the neural response waveform within each cycle match the time scales of the stimulus rhythm. Therefore, neural entrainment at very low frequency implies not only that the neural response repeats at f Hz but also that each period of the neural response is a slow wave matching the time scale of a f Hz sinusoid.

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

Perception is an active inferential process in which prior knowledge is combined with sensory input, the result of which determines the contents of awareness. Accordingly, previous experience is known to help the brain "decide" what to perceive. However, a critical aspect that has not been addressed is that previous experience can exert 2 opposing effects on perception: An attractive effect, sensitizing the brain to perceive the same again (hysteresis), or a repulsive effect, making it more likely to perceive something else (adaptation). We used functional magnetic resonance imaging and modeling to elucidate how the brain entertains these 2 opposing processes, and what determines the direction of such experience-dependent perceptual effects. We found that although affecting our perception concurrently, hysteresis and adaptation map into distinct cortical networks: a widespread network of higher-order visual and fronto-parietal areas was involved in perceptual stabilization, while adaptation was confined to early visual areas. This areal and hierarchical segregation may explain how the brain maintains the balance between exploiting redundancies and staying sensitive to new information. We provide a Bayesian model that accounts for the coexistence of hysteresis and adaptation by separating their causes into 2 distinct terms: Hysteresis alters the prior, whereas adaptation changes the sensory evidence (the likelihood function).

Upon repetition, certain stimuli induce reduced neural responses (i.e., repetition suppression), whereas others evoke stronger signals (i.e., repetition enhancement). It has been hypothesized that stimulus properties (e.g., visibility) determine the direction of the repetition effect. Here, we show that the very same stimuli can induce both repetition suppression and enhancement, whereby the only determining factor is the number of repetitions. Repeating the same, initially novel low-visible pictures of scenes for up to 5 times enhanced the blood oxygen level-dependent (BOLD) response in scene-selective areas, that is, the parahippocampal place area (PPA) and the transverse occipital sulcus (TOS), presumably reflecting the strengthening of the internal representation. Additional repetitions (6-9) resulted in progressively attenuated neural responses indicating a more efficient representation of the now familiar stimulus. Behaviorally, repetition led to increasingly faster responses and higher visibility ratings. Novel scenes induced the largest BOLD response in the PPA and also higher activity in yet another scene-selective region, the retrospenial cortex (RSC). We propose that 2 separable processes modulate activity in the PPA: one process optimizes the internal stimulus representation and involves TOS and the other differentiates between familiar and novel scenes and involves RSC.

Whether an object captures our attention depends on its bottom-up salience, that is, how different it is compared with its neighbors, and top-down control, that is, our current inner goals. At which neuronal stage they interact to guide behavior is still unknown. In a functional magnetic resonance imaging study, we found evidence for a hierarchy of saliency maps in human early visual cortex (V1 to hV4) and identified where bottom-up saliency interacts with top-down control: V1 represented pure bottom-up signals, V2 was only responsive to top-down modulations, and in hV4 bottom-up saliency and top-down control converged. Two distinct cerebral networks exerted top-down control: distractor suppression engaged the left intraparietal sulcus, while target enhancement involved the frontal eye field and lateral occipital cortex. Hence, attentional selection is implemented in integrated maps in visual cortex, which provide precise topographic information about target-distractor locations thus allowing for successful visual search.

Which neural processes underlie our conscious experience? One theoretical view argues that the neural correlates of consciousness (NCC) reside in local activity in sensory cortices. Accordingly, local category-specific gamma band responses in visual cortex correlate with conscious perception. However, as most studies manipulated conscious perception by altering the amount of sensory evidence, it is possible that they reflect prerequisites or consequences of consciousness rather than the actual NCC. Here we directly address this issue by developing a new experimental paradigm in which conscious perception is modulated either by sensory evidence or by previous exposure of the images while recording intracranial EEG from the higher-order visual cortex of human epilepsy patients. A clear prediction is that neural processes directly reflecting conscious perception should be present regardless of how it comes about. In contrast, we observed that although subjective reports were modulated both by sensory evidence and by previous exposure, gamma band responses solely reflected sensory evidence. This result contradicts the proposal that local gamma band responses in the higher-order visual cortex reflect conscious perception.

Spoken sentence comprehension relies on rapid and effortless temporal integration of speech units displayed at different rates. Temporal integration refers to how chunks of information perceived at different time scales are linked together by the listener in mapping speech sounds onto meaning. The neural implementation of this integration remains unclear. This study explores the role of short and long windows of integration in accessing meaning from long samples of speech. In a cross-linguistic study, we explore the time course of oscillatory brain activity between 1 and 100 Hz, recorded using EEG, during the processing of native and foreign languages. We compare oscillatory responses in a group of Italian and Spanish native speakers while they attentively listen to Italian, Japanese, and Spanish utterances, played either forward or backward. The results show that both groups of participants display a significant increase in gamma band power (55-75 Hz) only when they listen to their native language played forward. The increase in gamma power starts around 1000 msec after the onset of the utterance and decreases by its end, resembling the time course of access to meaning during speech perception. In contrast, changes in low-frequency power show similar patterns for both native and foreign languages. We propose that gamma band power reflects a temporal binding phenomenon concerning the coordination of neural assemblies involved in accessing meaning of long samples of speech.

Solving the problem of consciousness remains one of the biggest challenges in modern science. One key step towards understanding consciousness is to empirically narrow down neural processes associated with the subjective experience of a particular content. To unravel these neural correlates of consciousness (NCC) a common scientific strategy is to compare perceptual conditions in which consciousness of a particular content is present with those in which it is absent, and to determine differences in measures of brain activity (the so called "contrastive analysis"). However, this comparison appears not to reveal exclusively the NCC, as the NCC proper can be confounded with prerequisites for and consequences of conscious processing of the particular content. This implies that previous results cannot be unequivocally interpreted as reflecting the neural correlates of conscious experience. Here we review evidence supporting this conjecture and suggest experimental strategies to untangle the NCC from the prerequisites and consequences of conscious experience in order to further develop the otherwise valid and valuable contrastive methodology.