Faculty Bibliography

Visual area V2 of the primate cortex receives the largest projection from area V1. V2 is thought to use its striate inputs as the basis for computations that are important for visual form processing, such as signaling angles, object borders, illusory contours, and relative binocular disparity. However, it remains unclear how selectivity for these stimulus properties emerges in V2, in part because the functional properties of the inputs are unknown. We used antidromic electrical stimulation to identify V1 neurons that project directly to V2 (10% of all V1 neurons recorded) and characterized their electrical and visual responses. V2-projecting neurons were concentrated in the superficial and middle layers of striate cortex, consistent with the known anatomy of this cortico-cortical circuit. Most were fast conducting and temporally precise in their electrical responses, and had broad spike waveforms consistent with pyramidal regular-spiking excitatory neurons. Overall, projection neurons were functionally diverse. Most, however, were tuned for orientation and binocular disparity and were strongly suppressed by large stimuli. Projection neurons included those selective and invariant to spatial phase, with roughly equal proportions. Projection neurons found in superficial layers had longer conduction times, broader spike waveforms, and were more responsive to chromatic stimuli; those found in middle layers were more strongly selective for motion direction and binocular disparity. Collectively, these response properties may be well suited for generating complex feature selectivity in and beyond V2.

There is no generally accepted account of the function of the second visual cortical area (V2), partly because no simple response properties robustly distinguish V2 neurons from those in primary visual cortex (V1). We constructed synthetic stimuli replicating the higher-order statistical dependencies found in natural texture images and used them to stimulate macaque V1 and V2 neurons. Most V2 cells responded more vigorously to these textures than to control stimuli lacking naturalistic structure; V1 cells did not. Functional magnetic resonance imaging (fMRI) measurements in humans revealed differences between V1 and V2 that paralleled the neuronal measurements. The ability of human observers to detect naturalistic structure in different types of texture was well predicted by the strength of neuronal and fMRI responses in V2 but not in V1. Together, these results reveal a particular functional role for V2 in the representation of natural image structure.

To locate visual objects, the brain combines information about retinal location and direction of gaze. Studies in monkeys have demonstrated that eye position modulates the gain of visual signals with "gain fields," so that single neurons represent both retinotopic location and eye position. We wished to know whether eye position and retinotopic stimulus location are both represented in human visual cortex. Using functional magnetic resonance imaging, we measured separately for each of several different gaze positions cortical responses to stimuli that varied periodically in retinal locus. Visually evoked responses were periodic following the periodic retinotopic stimulation. Only the response amplitudes depended on eye position; response phases were indistinguishable across eye positions. We used multivoxel pattern analysis to decode eye position from the spatial pattern of response amplitudes. The decoder reliably discriminated eye position in five of the early visual cortical areas by taking advantage of a spatially heterogeneous eye position-dependent modulation of cortical activity. We conclude that responses in retinotopically organized visual cortical areas are modulated by gain fields qualitatively similar to those previously observed neurophysiologically.

Many visual and auditory neurons have response properties that are well explained by pooling the rectified responses of a set of spatially shifted linear filters. These filters cannot be estimated using spike-triggered averaging (STA). Subspace methods such as spike-triggered covariance (STC) can recover multiple filters, but require substantial amounts of data, and recover an orthogonal basis for the subspace in which the filters reside rather than the filters themselves. Here, we assume a linear-nonlinear-linear-nonlinear (LN-LN) cascade model in which the first linear stage is a set of shifted ('convolutional') copies of a common filter, and the first nonlinear stage consists of rectifying scalar nonlinearities that are identical for all filter outputs. We refer to these initial LN elements as the 'subunits' of the receptive field. The second linear stage then computes a weighted sum of the responses of the rectified subunits. We present a method for directly fitting this model to spike data, and apply it to both simulated and real neuronal data from primate V1. The subunit model significantly outperforms STA and STC in terms of cross-validated accuracy and efficiency.

To explore the relative development of the dorsal and ventral extrastriate processing streams, we studied the development of sensitivity to form and motion in macaque monkeys (Macaca nemestrina). We used Glass patterns and random dot kinematograms (RDK) to assay ventral and dorsal stream function, respectively. We tested 24 animals, longitudinally or cross-sectionally, between the ages of 5 weeks and 3 years. Each animal was tested with Glass patterns and RDK stimuli with each of two pattern types--circular and linear--at each age using a two alternative forced-choice task. We measured coherence threshold for discrimination of the global form or motion pattern from an incoherent control stimulus. Sensitivity to global motion appeared earlier than to global form and was higher at all ages, but performance approached adult levels at similar ages. Infants were most sensitive to large spatial scale (Deltax) and fast speeds; sensitivity to fine scale and slow speeds developed more slowly independently of pattern type. Within the motion domain, pattern type had little effect on overall performance. However, within the form domain, sensitivity for linear Glass patterns was substantially poorer than that for concentric patterns. Our data show comparatively early onset for global motion integration ability, perhaps reflecting early development of the dorsal stream. However, both pathways mature over long time courses reaching adult levels between 2 and 3 years after birth.

Neurons in area MT are sensitive to the direction of motion of gratings and of plaids made by summing 2 gratings moving in different directions. MT component direction-selective (CDS) neurons respond to the individual gratings of a plaid. Pattern direction-selective (PDS) neurons on the other hand, combine component information and respond selectively to the resulting pattern motion. Adding a third grating creates a "triplaid," which contains 3 grating and 3 plaid motions and is perceptually multistable. To examine how direction-selective mechanisms parse the motion signals in triplaids, we recorded MT responses of anesthetized and awake macaques to stimuli in which 3 identical moving gratings whose directions were separated by 120 degrees were introduced in 3 successive epochs, going from grating to plaid to triplaid. CDS and PDS neurons-selected based on their responses to gratings and plaids-had strikingly different tuning properties in the triplaid epoch. CDS neurons were strongly tuned for the direction of motion of individual gratings, but PDS neurons nearly lost their selectivity for either the gratings or the plaids in the stimulus. We explain this reduced motion selectivity with a model that relates pattern selectivity of PDS neurons to a broad pooling of V1 afferents with a near-cosine weighting profile. Because PDS neurons signal both component and pattern motion in gratings and plaids, their reduced selectivity for motion in triplaids may be what makes these stimuli perceptually multistable.

Most neurons in cortical area MT (V5) are strongly direction selective, and their activity is closely associated with the perception of visual motion. These neurons have large receptive fields built by combining inputs with smaller receptive fields that respond to local motion. Humans integrate motion over large areas and can perceive what has been referred to as global motion. The large size and direction selectivity of MT receptive fields suggests that MT neurons may represent global motion. We have explored this possibility by measuring responses to a stimulus in which the directions of simultaneously presented local and global motion are independently controlled. Surprisingly, MT responses depended only on the local motion and were unaffected by the global motion. Yet, under similar conditions, human observers perceive global motion and are impaired in discriminating local motion. Although local motion perception might depend on MT signals, global motion perception depends on mechanisms qualitatively different from those in MT. Motion perception therefore does not depend on a single cortical area but reflects the action and interaction of multiple brain systems

Human and macaque observers can detect and discriminate visual forms defined by differences in texture. The neurophysiological correlates of visual texture perception are not well understood and have not been studied extensively at the single-neuron level in the primate brain. We used a novel family of texture patterns to measure the selectivity of neurons in extrastriate cortical area V2 of the macaque (Macaca nemestrina, Macaca fascicularis) for the orientation of texture-defined form, and to distinguish responses to luminance- and texture-defined form. Most V2 cells were selective for the orientation of luminance-defined form; they signaled the orientation of the component gratings that made up the texture patterns but not the overall pattern orientation. In some cells, these luminance responses were modulated by the direction or orientation of the texture envelope, suggesting an interaction of luminance and texture signals. We found little evidence for a 'cue-invariant' representation in monkey V2. Few cells showed selectivity for the orientation of texture-defined form; they signaled the orientation of the texture patterns and not that of the component gratings. Small datasets recorded in monkey V1 and cat area 18 showed qualitatively similar patterns of results. Consistent with human functional imaging studies, our findings suggest that signals related to texture-defined form in primate cortex are most salient in areas downstream of V2. V2 may still provide the foundation for texture perception, through the interaction of luminance- and texture-based signals

Visual function depends on the accuracy of signals carried by visual cortical neurons. Combining information across neurons should improve this accuracy because single neuron activity is variable. We examined the reliability of information inferred from populations of simultaneously recorded neurons in macaque primary visual cortex. We considered a decoding framework that computes the likelihood of visual stimuli from a pattern of population activity by linearly combining neuronal responses and tested this framework for orientation estimation and discrimination. We derived a simple parametric decoder assuming neuronal independence and a more sophisticated empirical decoder that learned the structure of the measured neuronal response distributions, including their correlated variability. The empirical decoder used the structure of these response distributions to perform better than its parametric variant, indicating that their structure contains critical information for sensory decoding. These results show how neuronal responses can best be used to inform perceptual decision-making