Faculty Bibliography

Vision relies on both specific knowledge of visual attributes, such as object categories, and general brain states, such as those reflecting arousal. We hypothesized that these phenomena independently influence recognition of forthcoming stimuli through distinct processes reflected in spontaneous neural activity. Here, we recorded magnetoencephalographic (MEG) activity in participants (N = 24) who viewed images of objects presented at recognition threshold. Using multivariate analysis applied to sensor-level activity patterns recorded before stimulus presentation, we identified two neural processes influencing subsequent subjective recognition: a general process, which disregards stimulus category and correlates with pupil size, and a specific process, which facilitates category-specific recognition. The two processes are doubly-dissociable: the general process correlates with changes in criterion but not in sensitivity, whereas the specific process correlates with changes in sensitivity but not in criterion. Our findings reveal distinct mechanisms of how spontaneous neural activity influences perception and provide a framework to integrate previous findings.

Cortical responses to sensory inputs vary across repeated presentations of identical stimuli, but how this trial-to-trial variability impacts detection of sensory inputs is not fully understood. Using multi-channel local field potential (LFP) recordings in primary somatosensory cortex (S1) of the awake mouse, we optimized a data-driven cortical state classifier to predict single-trial sensory-evoked responses, based on features of the spontaneous, ongoing LFP recorded across cortical layers. Our findings show that, by utilizing an ongoing prediction of the sensory response generated by this state classifier, an ideal observer improves overall detection accuracy and generates robust detection of sensory inputs across various states of ongoing cortical activity in the awake brain, which could have implications for variability in the performance of detection tasks across brain states.

Past experiences have enormous power in shaping our daily perception. Currently, dynamical neural mechanisms underlying this process remain mysterious. Exploiting a dramatic visual phenomenon, where a single experience of viewing a clear image allows instant recognition of a related degraded image, we investigated this question using MEG and 7 Tesla fMRI in humans. We observed that following the acquisition of perceptual priors, different degraded images are represented much more distinctly in neural dynamics starting from ~500 ms after stimulus onset. Content-specific neural activity related to stimulus-feature processing dominated within 300 ms after stimulus onset, while content-specific neural activity related to recognition processing dominated from 500 ms onward. Model-driven MEG-fMRI data fusion revealed the spatiotemporal evolution of neural activities involved in stimulus, attentional, and recognition processing. Together, these findings shed light on how experience shapes perceptual processing across space and time in the brain.

Brain electric field potentials are dominated by an arrhythmic broadband signal, but the underlying mechanism is poorly understood. Here we propose that broadband power spectra characterize recurrent neural networks of nodes (neurons or clusters of neurons), endowed with an effective balance between excitation and inhibition tuned to keep the network on the edge of dynamical instability. These networks show a fast mode reflecting local dynamics and a slow mode emerging from distributed recurrent connections. Together, the 2 modes produce power spectra similar to those observed in human intracranial EEG (i.e., electrocorticography, ECoG) recordings. Moreover, such networks convert spatial input correlations across nodes into temporal autocorrelation of network activity. Consequently, increased independence between nodes reduces low-frequency power, which may explain changes observed during behavioral tasks. Lastly, varying network coupling causes activity changes that resemble those observed in human ECoG across different arousal states. The model links macroscopic features of empirical ECoG power to a parsimonious underlying network structure, and suggests mechanisms for changes observed across behavioral and arousal states. This work provides a computational framework to generate and test hypotheses about cellular and network mechanisms underlying whole brain electrical dynamics, their variations across states, and potential alterations in brain diseases.

In a near-threshold sensory detection task, an animal sometimes detects and sometimes misses the same physical stimulus. A simple hypothesis is that perceptual variability is linked to variability in sensory-evoked responses in the brain as early as primary cortex. Response variability arises in part from the interaction of sensory (Figure presented) inputs with ongoing activity and is partially predictable based on the pre-stimulus cortical state. If variability in evoked responses is linked to perception, and if that variability is predictable, we would expect that it would be possible to predict based on ongoing activity whether sensory cortex is primed to detect a sensory input. Here, we determine the pre-stimulus features that are predictive of variability in the evoked response in the awake animal. We then ask what implications these observations have for the detectability of a stimulus. Using data obtained from multi-electrode recordings across the cortical depth in S1 of awake mice, we systematically quantify how much variability in the sensory-evoked LFP response is predictable from ongoing LFP activity (Fig. 1AB). This interaction has been studied extensively in the anesthetized animal [e.g., 1, 2], where the major predictors of response variability are the degree of cortical synchronization, quantified by the amount of low-frequency power, and the phase of low-frequency oscillations at which sensory input occurred. Similarly, we found that the degree of synchronization was predictive, but instead of oscillation phase, the instantaneous level of activation of the LFP in layer 4 was a useful predictor. Specifically, positive excursions in the LFP and more low-frequency (1-5 Hz) power in the LFP in the pre-stimulus period predicted larger sensory-evoked responses ("high-response state"). Using a regularized estimator of current-source density (CSD) [3] on single trials, we localized the most predictive ongoing signal to a current source location near layer 4. Finally, we found that no significant predictive power was gained by increasing the complexity of the decoder or by utilizing the full array of channels. Thus, the most predictive signatures of ongoing activity are remarkably simple and could be accessible to downstream areas. Next, we examined the impact of predictable variability on an ideal observer analysis of the detectability of sensory events (Fig. 1C). We built a detection model, in which the detection threshold is either fixed, or adaptive and based on the pre-stimulus features that are predictive of evoked variability. We quantified the accuracy of the model in terms of the simulated hit rate and the false alarm rate. Detection was more accurate in the adaptive threshold model. In the fixed-threshold model, pre-stimulus features predicted hit and miss trials. This relationship was weaker in the adaptive- threshold model, where hits as well as false alarms were nearly equally as likely to occur in low- or high-response state. In summary, if sensory perception is built on the cortical response and variability in this response is completely unpredictable, then perceptual variability would to some extent be determined by cortical variability. However, if cortical variability is predictable and downstream circuits in the brain make this prediction, then the perceptual variability could be decoupled from cortical variability

The vast majority of experiments examining perception and behavior are conducted using experimental paradigms which adhere to a rigid trial structure -- each trial consists of a brief and discrete series of events, and is regarded as independent from all other trials. The assumptions underlying this structure ignore the reality that natural behavior is rarely discrete, brain activity follows multiple time courses which do not necessarily conform to the trial structure, and the natural environment has statistical structure and dynamics that exhibit long-range temporal correlation. Modern advances in statistical modeling and analysis offer tools that make it feasible for experiments to move beyond the rigid independent and identically distributed trial structure. Here we review literature that serves as evidence for the feasibility and advantages of moving beyond trial-based paradigms in order to understand the neural basis of perception and cognition. Furthermore, we propose a synthesis of these efforts, integrating the characterization of natural stimulus properties with measurements of continuous neural activity and behavioral outputs within the framework of sensory-cognitive-motor-loops. Such a framework provides a basis for the study of natural statistics, naturalistic tasks, and/or slow fluctuations in brain activity, which should provide starting points for important generalizations of analytical tools in neuroscience and subsequent progress in understanding the neural basis of perception and cognition.

How prior knowledge shapes perceptual processing across the human brain, particularly in the frontoparietal (FPN) and default-mode (DMN) networks, remains unknown. Using ultra-high-field (7T) functional magnetic resonance imaging (fMRI), we elucidated the effects that the acquisition of prior knowledge has on perceptual processing across the brain. We observed that prior knowledge significantly impacted neural representations in the FPN and DMN, rendering responses to individual visual images more distinct from each other, and more similar to the image-specific prior. In addition, neural representations were structured in a hierarchy that remained stable across perceptual conditions, with early visual areas and DMN anchored at the two extremes. Two large-scale cortical gradients occur along this hierarchy: first, dimensionality of the neural representational space increased along the hierarchy; second, prior's impact on neural representations was greater in higher-order areas. These results reveal extensive and graded influences of prior knowledge on perceptual processing across the brain.

While neuroscientific research on perceptual awareness has traditionally focused on the spatial and temporal localizations of neural activity underlying conscious processing, recent development suggests that the dynamic characteristics of spatiotemporally distributed neural activity contain important clues about the neural computational mechanisms underlying conscious processing. Here, we summarize recent progress.

Forming valid predictions about the environment is crucial to survival. However, whether humans are able to form valid predictions about natural stimuli based on their temporal statistical regularities remains unknown. Here we presented subjects with tone sequences whose pitch fluctuation over time capture long-range temporal dependence structures prevalent in natural stimuli. We found that subjects were able to exploit such naturalistic statistical regularities to make valid predictions about upcoming items in a sequence. Magnetoencephalography (MEG) recordings revealed that slow, arrhythmic cortical dynamics tracked the evolving pitch sequence over time such that neural activity at a given moment was influenced by the pitch of up to seven previous tones. Importantly, such history integration contained in neural activity predicted the expected pitch of the upcoming tone, providing a concrete computational mechanism for prediction. These results establish humans' ability to make valid predictions based on temporal regularities inherent in naturalistic stimuli and further reveal the neural mechanisms underlying such predictive computation.SIGNIFICANCE STATEMENTA fundamental question in neuroscience is how the brain predicts upcoming events in the environment. To date, this question has primarily been addressed in experiments using relatively simple stimulus sequences. Here, we study predictive processing in the human brain using auditory tone sequences that exhibit temporal statistical regularities similar to those found in natural stimuli. We observed that humans are able to form valid predictions based on such complex temporal statistical regularities. We further show that neural response to a given tone in the sequence reflects integration over the preceding tone sequence, and that this history dependence forms the foundation for prediction. These findings deepen our understanding of how humans form predictions in an ecologically valid environment.