Faculty Bibliography

Many visual and auditory neurons have response properties that are well explained by pooling the rectified responses of a set of spatially shifted linear filters. These filters cannot be estimated using spike-triggered averaging (STA). Subspace methods such as spike-triggered covariance (STC) can recover multiple filters, but require substantial amounts of data, and recover an orthogonal basis for the subspace in which the filters reside rather than the filters themselves. Here, we assume a linear-nonlinear-linear-nonlinear (LN-LN) cascade model in which the first linear stage is a set of shifted ('convolutional') copies of a common filter, and the first nonlinear stage consists of rectifying scalar nonlinearities that are identical for all filter outputs. We refer to these initial LN elements as the 'subunits' of the receptive field. The second linear stage then computes a weighted sum of the responses of the rectified subunits. We present a method for directly fitting this model to spike data, and apply it to both simulated and real neuronal data from primate V1. The subunit model significantly outperforms STA and STC in terms of cross-validated accuracy and efficiency.

To explore the relative development of the dorsal and ventral extrastriate processing streams, we studied the development of sensitivity to form and motion in macaque monkeys (Macaca nemestrina). We used Glass patterns and random dot kinematograms (RDK) to assay ventral and dorsal stream function, respectively. We tested 24 animals, longitudinally or cross-sectionally, between the ages of 5 weeks and 3 years. Each animal was tested with Glass patterns and RDK stimuli with each of two pattern types--circular and linear--at each age using a two alternative forced-choice task. We measured coherence threshold for discrimination of the global form or motion pattern from an incoherent control stimulus. Sensitivity to global motion appeared earlier than to global form and was higher at all ages, but performance approached adult levels at similar ages. Infants were most sensitive to large spatial scale (Deltax) and fast speeds; sensitivity to fine scale and slow speeds developed more slowly independently of pattern type. Within the motion domain, pattern type had little effect on overall performance. However, within the form domain, sensitivity for linear Glass patterns was substantially poorer than that for concentric patterns. Our data show comparatively early onset for global motion integration ability, perhaps reflecting early development of the dorsal stream. However, both pathways mature over long time courses reaching adult levels between 2 and 3 years after birth.

Neurons in area MT are sensitive to the direction of motion of gratings and of plaids made by summing 2 gratings moving in different directions. MT component direction-selective (CDS) neurons respond to the individual gratings of a plaid. Pattern direction-selective (PDS) neurons on the other hand, combine component information and respond selectively to the resulting pattern motion. Adding a third grating creates a "triplaid," which contains 3 grating and 3 plaid motions and is perceptually multistable. To examine how direction-selective mechanisms parse the motion signals in triplaids, we recorded MT responses of anesthetized and awake macaques to stimuli in which 3 identical moving gratings whose directions were separated by 120 degrees were introduced in 3 successive epochs, going from grating to plaid to triplaid. CDS and PDS neurons-selected based on their responses to gratings and plaids-had strikingly different tuning properties in the triplaid epoch. CDS neurons were strongly tuned for the direction of motion of individual gratings, but PDS neurons nearly lost their selectivity for either the gratings or the plaids in the stimulus. We explain this reduced motion selectivity with a model that relates pattern selectivity of PDS neurons to a broad pooling of V1 afferents with a near-cosine weighting profile. Because PDS neurons signal both component and pattern motion in gratings and plaids, their reduced selectivity for motion in triplaids may be what makes these stimuli perceptually multistable.

Most neurons in cortical area MT (V5) are strongly direction selective, and their activity is closely associated with the perception of visual motion. These neurons have large receptive fields built by combining inputs with smaller receptive fields that respond to local motion. Humans integrate motion over large areas and can perceive what has been referred to as global motion. The large size and direction selectivity of MT receptive fields suggests that MT neurons may represent global motion. We have explored this possibility by measuring responses to a stimulus in which the directions of simultaneously presented local and global motion are independently controlled. Surprisingly, MT responses depended only on the local motion and were unaffected by the global motion. Yet, under similar conditions, human observers perceive global motion and are impaired in discriminating local motion. Although local motion perception might depend on MT signals, global motion perception depends on mechanisms qualitatively different from those in MT. Motion perception therefore does not depend on a single cortical area but reflects the action and interaction of multiple brain systems

Human and macaque observers can detect and discriminate visual forms defined by differences in texture. The neurophysiological correlates of visual texture perception are not well understood and have not been studied extensively at the single-neuron level in the primate brain. We used a novel family of texture patterns to measure the selectivity of neurons in extrastriate cortical area V2 of the macaque (Macaca nemestrina, Macaca fascicularis) for the orientation of texture-defined form, and to distinguish responses to luminance- and texture-defined form. Most V2 cells were selective for the orientation of luminance-defined form; they signaled the orientation of the component gratings that made up the texture patterns but not the overall pattern orientation. In some cells, these luminance responses were modulated by the direction or orientation of the texture envelope, suggesting an interaction of luminance and texture signals. We found little evidence for a 'cue-invariant' representation in monkey V2. Few cells showed selectivity for the orientation of texture-defined form; they signaled the orientation of the texture patterns and not that of the component gratings. Small datasets recorded in monkey V1 and cat area 18 showed qualitatively similar patterns of results. Consistent with human functional imaging studies, our findings suggest that signals related to texture-defined form in primate cortex are most salient in areas downstream of V2. V2 may still provide the foundation for texture perception, through the interaction of luminance- and texture-based signals

Visual function depends on the accuracy of signals carried by visual cortical neurons. Combining information across neurons should improve this accuracy because single neuron activity is variable. We examined the reliability of information inferred from populations of simultaneously recorded neurons in macaque primary visual cortex. We considered a decoding framework that computes the likelihood of visual stimuli from a pattern of population activity by linearly combining neuronal responses and tested this framework for orientation estimation and discrimination. We derived a simple parametric decoder assuming neuronal independence and a more sophisticated empirical decoder that learned the structure of the measured neuronal response distributions, including their correlated variability. The empirical decoder used the structure of these response distributions to perform better than its parametric variant, indicating that their structure contains critical information for sensory decoding. These results show how neuronal responses can best be used to inform perceptual decision-making

Amblyopia is usually associated with the presence of anisometropia, strabismus or both early in life. We set out to explore quantitative relationships between the degree of anisometropia and the loss of visual function, and to examine how the presence of strabismus affects visual function in observers with anisometropia. We measured optotype acuity, Pelli-Robson contrast sensitivity and stereoacuity in 84 persons with anisometropia and compared their results with those of 27 persons with high bilateral refractive error (isoametropia) and 101 persons with both strabismus and anisometropia. All subjects participated in a large-scale study of amblyopia (McKee et al., 2003). We found no consistent visual abnormalities in the strong eye, and therefore report only on vision in the weaker, defined as the eye with lower acuity. LogMAR acuity falls off markedly with increasing anisometropia in non-strabismic anisometropes, while contrast sensitivity is much less affected. Acuity degrades rapidly with increases in both hyperopic and myopic anisometropia, but the risk of amblyopia is about twice as great in hyperopic than myopic anisometropes of comparable refractive imbalance. For a given degree of refractive imbalance, strabismic anisometropes perform considerably worse than anisometropes without strabismus--visual acuity for strabismics was on average 2.5 times worse than for non-strabismics with similar anisometropia. For observers with equal refractive error in the two eyes there is very little change in acuity or sensitivity with increasing (bilateral) refractive error except for one extreme individual (bilaterally refractive error of -15 D). Most pure anisometropes with interocular differences less than 4D retain some stereopsis, and the degree is correlated with the acuity of the weak eye. We conclude that even modest interocular differences in refractive error can influence visual function

We investigated how the human visual system and the pursuit system react to visual motion noise. We presented three different types of random-dot kinematograms at five different coherence levels. For transparent motion, the signal and noise labels on each dot were preserved throughout each trial, and noise dots moved with the same speed as the signal dots but in fixed random directions. For white noise motion, every 20 ms the signal and noise labels were randomly assigned to each dot and noise dots appeared at random positions. For Brownian motion, signal and noise labels were also randomly assigned, but the noise dots moved at the signal speed in a direction that varied randomly from moment to moment. Neither pursuit latency nor early eye acceleration differed among the different types of kinematograms. Late acceleration, pursuit gain, and perceived speed all depended on kinematogram type, with good agreement between pursuit gain and perceived speed. For transparent motion, pursuit gain and perceived speed were independent of coherence level. For white and Brownian motions, pursuit gain and perceived speed increased with coherence but were higher for white than for Brownian motion. This suggests that under our conditions, the pursuit system integrates across all directions of motion but not across all speeds

In human and non-human primates, higher form vision matures substantially later than spatial acuity and contrast sensitivity, as revealed by performance on such tasks as figure-ground segregation and contour integration. Our goal was to understand whether delayed maturation on these tasks was intrinsically form-dependent or, rather, related to the nature of spatial integration necessary for extracting task-relevant cues. We used an intermediate-level form task that did not call for extensive spatial integration. We trained monkeys (6-201 weeks) to discriminate the orientation of pattern modulation in a two-alternative forced choice paradigm. We presented two families of form patterns, defined by texture or contrast variations, and luminance-defined patterns for comparison. Infant monkeys could discriminate texture- and contrast-defined form as early as 6 weeks; sensitivity improved up to 40 weeks. Surprisingly, sensitivity for texture- and contrast-defined form matured earlier than for luminance-defined form. These results suggest that intermediate-level form vision develops in concert with basic spatial vision rather than following sequentially. Comparison with earlier results reveals that different aspects of form vision develop over different time courses, with processes that depend on comparing local image content maturing earlier than those requiring 'global' linking of multiple visual elements across a larger spatial extent