Faculty Bibliography

Over the course of the past decade, neurobiologists have become increasingly interested in concepts and models imported from economics. Terms such as "risk," "risk aversion," and "utility" have become commonplace in the neuroscientific literature as single-unit physiologists and human cognitive neuroscientists search for the biological correlates of economic theories of value and choice. Among neuroscientists, an incomplete understanding of these concepts has, however, led to a growing confusion that threatens to check the rapid advances in this area. Adding to the confusion, notions of risk have more recently been imported from finance, which employs quite different, although formally related, mathematical tools. Of course, the mixing of economic, financial, and neuroscientific traditions can only be beneficial in the long run, but truly understanding the conceptual machinery of each area is a prerequisite for obtaining that benefit. With that in mind, I present here an overview of economic and financial notions of risk and decision. The article begins with an overview of the classical economic approach to risk, as developed by Bernoulli. It then explains the important differences between the classical tradition and modern neoclassical economic approaches to these same concepts. Finally, I present a very brief overview of the financial tradition and its relation to the economic tradition. For novices, this should provide a reasonable introduction to concepts ranging from "risk aversion" to "risk premiums."

Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions.

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions--particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex--tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

Work in behaving primates indicates that midbrain dopamine neurons encode a prediction error, the difference between an obtained reward and the reward expected. Studies of dopamine action potential timing in the alert and anesthetized rat indicate that dopamine neurons respond in tonic and phasic modes, a distinction that has been less well characterized in the primates. We used spike train models to examine the relationship between the tonic and burst modes of activity in dopamine neurons while monkeys were performing a reinforced visuo-saccadic movement task. We studied spiking activity during four task-related intervals; two of these were intervals during which no task-related events occurred, whereas two were periods marked by task-related phasic activity. We found that dopamine neuron spike trains during the intervals when no events occurred were well described as tonic. Action potentials appeared to be independent, to occur at low frequency, and to be almost equally well described by Gaussian and Poisson-like (gamma) processes. Unlike in the rat, interspike intervals as low as 20 ms were often observed during these presumptively tonic epochs. Having identified these periods of presumptively tonic activity, we were able to quantitatively define phasic modulations (both increases and decreases in activity) during the intervals in which task-related events occurred. This analysis revealed that the phasic modulations of these neurons include both bursting, as has been described previously, and pausing. Together bursts and pauses seemed to provide a continuous, although nonlinear, representation of the theoretically defined reward prediction error of reinforcement learning.

Electrophysiological studies in monkeys have identified effector-related regions in the posterior parietal cortex (PPC). The lateral intraparietal area, for example, responds preferentially for saccades, whereas the parietal reach region responds preferentially for arm movements. However, the degree of effector selectivity actually observed is limited; each area contains neurons selective for the nonpreferred effector, and many neurons in both areas respond for both effectors. We used functional magnetic resonance imaging to assess the degree of effector preference at the population level, focusing on topographically organized regions in the human PPC [visual area V7, intraparietal sulcus 1 (IPS1), and IPS2]. An event-related design adapted from monkey experiments was used. In each trial, an effector cue preceded the appearance of a spatial target, after which a Go signal instructed subjects to produce the specified movement with the specified effector. Our results show that the degree of effector specificity is limited in many cortical areas and transitions gradually from saccade to reach preference as one moves through the hierarchy of areas in the occipital, parietal, and frontal cortices. Saccade preference was observed in visual cortex, including early areas and V7. IPS1 exhibited balanced activation to saccades and reaches, whereas IPS2 showed a weak but significant preference for reaches. In frontal cortex, areas near the central sulcus showed a clear and absolute preference for reaches, whereas the frontal eye field showed little or no effector selectivity. Although these results contradict many theoretical conclusions about effector specificity, they are compatible with the complex picture arising from electrophysiological studies and also with previous imaging studies that reported mostly overlapping saccade- and arm-related activation. The results are also compatible with theories of efficient coding in cortex.