Faculty Bibliography

Many neurons in visual cortical area MT signal the direction of motion of complex visual patterns, such as plaids composed of two superimposed drifting gratings. To compute the direction of pattern motion, MT neurons combine component motion signals over time and space. To determine the spatial and temporal limits of signal integration, we measured the responses of single MT neurons to a novel set of "pseudoplaid" stimuli in which the component gratings were alternated in time or space. As the temporal or spatial separation of the component gratings increased, neuronal selectivity for the direction of pattern motion decreased. Using descriptive models of signal integration, we inferred the temporal and spatial structure of the mechanisms that compute pattern direction selectivity. The median time constant for integration was roughly 10 ms, a timescale characteristic of integration by single cortical pyramidal neurons. The median spatial integration field was roughly one-third of the MT receptive field diameter, suggesting that the spatial limits are set by stages of processing in earlier areas of visual cortex where receptive fields are smaller than in MT. Interestingly, pattern direction-selective neurons had shorter temporal integration times than component direction-selective neurons but similar spatial integration windows. We conclude that pattern motion can only be signaled by MT neurons when the component motion signals co-occur within relatively narrow spatial and temporal limits. We interpret these results in the framework of recent hierarchical models of MT.

Big data has transformed fields such as physics and genomics. Neuroscience is set to collect its own big data sets, but to exploit its full potential, there need to be ways to standardize, integrate and synthesize diverse types of data from different levels of analysis and across species. This will require a cultural shift in sharing data across labs, as well as to a central role for theorists in neuroscience research.

Single neurons in areas V1 and V2 of macaque visual cortex respond selectively to luminance-modulated stimuli. These responses are often influenced by context, for example when stimuli extend outside the classical receptive field (CRF). These contextual phenomena, observed in many sensory areas, reflect a fundamental cortical computation and may inform perception by signaling second-order visual features which are defined by spatial relationships of contrast, orientation and spatial frequency. In the anesthetized, paralyzed macaque, we measured single-unit responses to a drifting preferred sinusoidal grating; low spatial frequency sinusoidal contrast modulations were applied to the grating, creating contrast-modulated, second-order forms. Most neurons responded selectively to the orientation of the contrast modulation of the preferred grating and were therefore second-order orientation-selective. Second-order selectivity was created by the asymmetric spatial organization of the excitatory CRF and suppressive extraclassical surround. We modeled these receptive field subregions using spatial Gaussians, sensitive to the modulation of contrast (not luminance) of the preferred carrier grating, that summed linearly and were capable of recovering asymmetrical receptive field organizations. Our modeling suggests that second-order selectivity arises both from elongated excitatory CRFs, asymmetrically organized extraclassical surround suppression, or both. We validated the model by successfully testing its predictions against conventional surround suppression measurements and spike-triggered analysis of second-order form responses. Psychophysical adaptation measurements on human observers revealed a pattern of second-order form selectivity consistent with neural response patterns. We therefore propose that cortical cells in primates do double duty, providing signals about both first- and second-order forms.

Responses of sensory neurons differ across repeated measurements. This variability is usually treated as stochasticity arising within neurons or neural circuits. However, some portion of the variability arises from fluctuations in excitability due to factors that are not purely sensory, such as arousal, attention and adaptation. To isolate these fluctuations, we developed a model in which spikes are generated by a Poisson process whose rate is the product of a drive that is sensory in origin and a gain summarizing stimulus-independent modulatory influences on excitability. This model provides an accurate account of response distributions of visual neurons in macaque lateral geniculate nucleus and cortical areas V1, V2 and MT, revealing that variability originates in large part from excitability fluctuations that are correlated over time and between neurons, and that increase in strength along the visual pathway. The model provides a parsimonious explanation for observed systematic dependencies of response variability and covariability on firing rate.

The perception of complex visual patterns emerges from neuronal activity in a cascade of areas in the primate cerebral cortex. We have probed the early stages of this cascade with "naturalistic" texture stimuli designed to capture key statistical features of natural images. Humans can recognize and classify these synthetic images and are insensitive to distortions that do not alter the local values of these statistics. The responses of neurons in the primary visual cortex, V1, are relatively insensitive to the statistical information in these textures. However, in the area immediately downstream, V2, cells respond more vigorously to these stimuli than to matched control stimuli. Humans show blood-oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI responses in V1 and V2) that are consistent with the neuronal measurements in macaque. These fMRI measurements, as well as neurophysiological work by others, show that true natural scenes become a more prominent driving feature of cortex downstream from V2. These results suggest a framework for thinking about how information about elementary visual features is transformed into the specific representations of scenes and objects found in areas higher in the visual pathway.

The notion of a set of neurons that form a "bridge locus" serving as the immediate substrate of visual perception is examined in the light of evidence on the architecture of the visual pathway, of current thinking about perceptual representations, and of the basis of perceptual awareness. The bridge locus is likely to be part of a tangled web of representations, and this complexity raises the question of whether another scheme that relies less on geography might offer a better framework. The bridge locus bears a close relationship to the neural correlate of consciousness (NCC), and like the NCC may be a concept which is no longer precise enough to provide a useful basis for reasoning about the relationship between brain activity and perceptual experience.

Visual area V2 of the primate cortex receives the largest projection from area V1. V2 is thought to use its striate inputs as the basis for computations that are important for visual form processing, such as signaling angles, object borders, illusory contours, and relative binocular disparity. However, it remains unclear how selectivity for these stimulus properties emerges in V2, in part because the functional properties of the inputs are unknown. We used antidromic electrical stimulation to identify V1 neurons that project directly to V2 (10% of all V1 neurons recorded) and characterized their electrical and visual responses. V2-projecting neurons were concentrated in the superficial and middle layers of striate cortex, consistent with the known anatomy of this cortico-cortical circuit. Most were fast conducting and temporally precise in their electrical responses, and had broad spike waveforms consistent with pyramidal regular-spiking excitatory neurons. Overall, projection neurons were functionally diverse. Most, however, were tuned for orientation and binocular disparity and were strongly suppressed by large stimuli. Projection neurons included those selective and invariant to spatial phase, with roughly equal proportions. Projection neurons found in superficial layers had longer conduction times, broader spike waveforms, and were more responsive to chromatic stimuli; those found in middle layers were more strongly selective for motion direction and binocular disparity. Collectively, these response properties may be well suited for generating complex feature selectivity in and beyond V2.

There is no generally accepted account of the function of the second visual cortical area (V2), partly because no simple response properties robustly distinguish V2 neurons from those in primary visual cortex (V1). We constructed synthetic stimuli replicating the higher-order statistical dependencies found in natural texture images and used them to stimulate macaque V1 and V2 neurons. Most V2 cells responded more vigorously to these textures than to control stimuli lacking naturalistic structure; V1 cells did not. Functional magnetic resonance imaging (fMRI) measurements in humans revealed differences between V1 and V2 that paralleled the neuronal measurements. The ability of human observers to detect naturalistic structure in different types of texture was well predicted by the strength of neuronal and fMRI responses in V2 but not in V1. Together, these results reveal a particular functional role for V2 in the representation of natural image structure.

To locate visual objects, the brain combines information about retinal location and direction of gaze. Studies in monkeys have demonstrated that eye position modulates the gain of visual signals with "gain fields," so that single neurons represent both retinotopic location and eye position. We wished to know whether eye position and retinotopic stimulus location are both represented in human visual cortex. Using functional magnetic resonance imaging, we measured separately for each of several different gaze positions cortical responses to stimuli that varied periodically in retinal locus. Visually evoked responses were periodic following the periodic retinotopic stimulation. Only the response amplitudes depended on eye position; response phases were indistinguishable across eye positions. We used multivoxel pattern analysis to decode eye position from the spatial pattern of response amplitudes. The decoder reliably discriminated eye position in five of the early visual cortical areas by taking advantage of a spatially heterogeneous eye position-dependent modulation of cortical activity. We conclude that responses in retinotopically organized visual cortical areas are modulated by gain fields qualitatively similar to those previously observed neurophysiologically.