Faculty Bibliography

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

The central goal of modern science that evolved during the Enlightenment was the empirical reduction of uncertainty by experimental inquiry. Although there have been challenges to this view in the physical sciences, where profoundly indeterminate events have been identified at the quantum level, the presumption that physical phenomena are fundamentally determinate seems to have defined modern behavioral science. Programs like those of the classical behaviorists, for example, were explicitly anchored to a fully deterministic worldview, and this anchoring clearly influenced the experiments that those scientists chose to perform. Recent advances in the psychological, social, and neural sciences, however, have caused a number of scholars to begin to question the assumption that all of behavior can be regarded as fundamentally deterministic in character. Although it is not yet clear whether the generative mechanisms for human and animal behavior will require a philosophically indeterminate approach, it is clear that behavioral scientists of all kinds are beginning to engage the issues of indeterminacy that plagued physics at the beginning of the twentieth century.

Economics, psychology, and neuroscience are converging today into a single, unified discipline with the ultimate aim of providing a single, general theory of human behavior. This is the emerging field of neuroeconomics in which consilience, the accordance of two or more inductions drawn from different groups of phenomena, seems to be operating. Economists and psychologists are providing rich conceptual tools for understanding and modeling behavior, while neurobiologists provide tools for the study of mechanism. The goal of this discipline is thus to understand the processes that connect sensation and action by revealing the neurobiological mechanisms by which decisions are made. This review describes recent developments in neuroeconomics from both behavioral and biological perspectives.

Behavioral studies suggest that making a decision involves representing the overall desirability of all available actions and then selecting that action that is most desirable. Physiological studies have proposed that neurons in the parietal cortex play a role in selecting movements for execution. To test the hypothesis that these parietal neurons encode the subjective desirability of making particular movements, we exploited Nash's game theoretic equilibrium, during which the subjective desirability of multiple actions should be equal for human players. Behavior measured during a strategic game suggests that monkeys' choices, like those of humans, are guided by subjective desirability. Under these conditions, activity in the parietal cortex was correlated with the relative subjective desirability of actions irrespective of the specific combination of reward magnitude, reward probability, and response probability associated with each action. These observations may help place many recent findings regarding the posterior parietal cortex into a common conceptual framework.

The substantia nigra pars reticulata (SNr), a major output nucleus of the basal ganglia, has been implicated anatomically, pharmacologically and physiologically in the generation of saccadic eye movements. However, the unique contribution of the SNr to saccade generation remains elusive. We studied the activity of SNr neurons while rhesus monkeys made saccades from different initial orbital positions, to determine what effects, if any, eye position had on SNr neuronal activity. We found that there was no effect of eye position on SNr neuronal responses. We also examined the responses of SNr neurons during memory-guided saccades to determine whether SNr discharges were affected by whether the target of the upcoming saccade was visible. We found that there was no change in response properties during memory saccade trials as compared to otherwise identical visually guided trials. SNr neurons appear to carry no information about either eye position or whether a movement is guided by a visible or remembered target. These results suggest that nigral signals are encoded in the same coordinate frame as those in the SC and FEF, but that unlike neuronal responses in these areas, SNr activity is not influenced by whether the saccade target remains visible until the movement is executed.

Introduction: It has been shown that subjects are faster and more accurate at detecting or discriminating stimuli when they are more certain of where a stimulus will appear. We have shown that a probability paradigm, in which observers use the probability of where a stimulus is likely to occur, can direct the allocation of resources and improve accuracy with increasing probability, or spatial certainty (Ciaramitaro et al, 2001). We now study the temporal dynamics of attention across probability transitions, to investigate how quickly observers track shifts in probability, as the certainty of where to attend changes. Method: Six observers performed an orientation discrimination task, viewing an extrafoveal stimulus (102ms) that was followed by a mask (102ms) after a delay of 17, 35 or 52ms. The probability of stimulus occurrence in the left or right hemifield switched between 20% and 80% in several blocks of ∼200 trials each. Behavioral data were convolved with a gaussian to derive a trial-by-trial running estimate of fluctuations in performance over time. Results & Conclusion: When probability transitions were signaled, observers' overall performance improved as probability increased across blocks, whereas when transitions were not signaled, and observers may have been less certain of the probability condition, their overall performance was not well matched to changes in probability. On a trial-by-trial basis, performance within a block was not always stable, potentially obscuring overall differences between blocks, and performance changes across blocks often showed rapid transitions, suggesting that observers learned the new probability quickly. Quantifying the dynamics of changes in behavior over time is an important step if we ultimately want to link such changes to dynamic changes at the neuronal level as we switch attention to different locations.