Faculty Bibliography

Decision-making is often viewed as a two-stage process, where subjective values are first assigned to each option and then the option of the highest value is selected. Converging evidence suggests that these subjective values are represented in the striatum and medial prefrontal cortex (MPFC). A separate line of evidence suggests that activation in the same areas represents the values of rewards even when choice is not required, as in classical conditioning tasks. However, it is unclear whether the same neural mechanism is engaged in both cases. To address this question we measured brain activation with functional magnetic resonance imaging while human subjects passively viewed individual consumer goods. We then sampled activation from predefined regions of interest and used it to predict subsequent choices between the same items made outside of the scanner. Our results show that activation in the striatum and MPFC in the absence of choice predicts subsequent choices, suggesting that these brain areas represent value in a similar manner whether or not choice is required.

The neurotransmitter dopamine is central to the emerging discipline of neuroeconomics; it is hypothesized to encode the difference between expected and realized rewards and thereby to mediate belief formation and choice. We develop the first formal test of this theory of dopaminergic function, based on a recent axiomatization by Caplin and Dean [2008A]. These tests are satisfied by neural activity in the nucleus accumbens, an area rich in dopamine receptors. We find evidence for separate positive and negative reward prediction error signals, suggesting that behavioral asymmetries in response to losses and gains may parallel asymmetries in nucleus accumbens activity.

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

Standard methods for behavioral and neurophysiological experiments in the non-human primate rely on controlled water access as a means for motivating subject performance. It is, however, still not clear whether animals are able to regulate their fluid balance appropriately under these experimental settings. Further, the physical state associated with a subject monkey's thirst has not yet been objectively assessed under these conditions. Both of these deficiencies arise from the lack of a method for independently evaluating the hydration state of these subjects during experimental testing. To address these limitations, we measured the blood osmolality, the most widely used hematological index of hydration status, of three rhesus monkeys under conditions of controlled water access while they participated in a standard reinforced behavioral task for fluid rewards. We found that day-to-day hydration levels, as measured by serum osmolality, appears to be well regulated in a narrow range of values (300-320 mOsmo/kg H(2)O) by experimental subjects under these conditions: animals work harder and longer to earn more water rewards on a day when they are in a lower hydration state (higher osmolality) than when they are in a higher hydration state (lower osmolality). We also found that osmolality level decreases almost immediately after water intake, within 30 min, in a surprisingly linear manner. Osmolality thus seems to provide a fairly precise reflection of the monkeys' hydration state on a timescale of minutes. This evidence suggests that osmolality can be used as a tool for monitoring the hydration level of experimental subjects.

Many decisions involve a trade-off between the quality of an outcome and the time at which that outcome is received. In psychology and behavioral economics, the most widely studied models hypothesize that the values of future gains decline as a roughly hyperbolic function of delay from the present. Recently, it has been proposed that this hyperbolic-like decline in value arises from the interaction of two separate neural systems: one specialized to value immediate rewards and the other specialized to value delayed rewards. Here we report behavioral and functional magnetic resonance imaging results that are inconsistent with both the standard behavioral models of discounting and the hypothesis that separate neural systems value immediate and delayed rewards. Behaviorally, we find that human subjects do not necessarily make the impulsive preference reversals predicted by hyperbolic-like discounting. We also find that blood oxygenation level dependent activity in ventral striatum, medial prefrontal, and posterior cingulate cortex does not track whether an immediate reward was present, as proposed by the separate neural systems hypothesis. Activity in these regions was correlated with the subjective value of both immediate and delayed rewards. Rather than encoding only the relative value of one reward compared with another, these values are represented on a more absolute scale. These data support an alternative behavioral-neural model (which we call "ASAP"), in which subjective value declines hyperbolically relative to the soonest currently available reward and a small number of valuation areas serve as a final common pathway through which these subjective values guide choice.

The mathematical formulations used to study the neurophysiological signals governing choice behavior fall under one of two major theoretical frameworks: "choice probability" or "subjective value." These two formulations represent behavioral quantities closely tied to the decision process, but it is unknown whether one of these variables, or both, dominates the neural mechanisms that mediate choice. Value and choice probability are difficult to distinguish in practice, because higher-valued options are chosen more frequently in free-choice tasks. This distinction is particularly relevant for sensorimotor areas such as parietal cortex, where both value information and motor signals related to choice have been observed. We recorded the activity of neurons in the lateral intraparietal area while monkeys performed an intertemporal choice task for rewards differing in delay to reinforcement. Here we show that the activity of parietal neurons is precisely correlated with the individual-specific discounted value of delayed rewards, with peak subjective value modulation occurring early in task trials. In contrast, late in the decision process these same neurons transition to encode the selected action. When directly compared, the strong delay-related modulation early during decision making is driven by subjective value rather than the monkey's probability of choice. These findings show that in addition to information about gains, parietal cortex also incorporates information about delay into a precise physiological correlate of economic value functions, independent of the probability of choice.

Risk and ambiguity are two conditions in which the consequences of possible outcomes are not certain. Under risk, the probabilities of different outcomes can be estimated, whereas under ambiguity, even these probabilities are not known. Although most people exhibit at least some aversion to both risk and ambiguity, the degree of these aversions is largely uncorrelated across subjects, suggesting that risk aversion and ambiguity aversion are distinct phenomena. Previous studies have shown differences in brain activations for risky and ambiguous choices and have identified neural mechanisms that may mediate transitions from conditions of ambiguity to conditions of risk. Unknown, however, is whether the value of risky and ambiguous options is necessarily represented by two distinct systems or whether a common mechanism can be identified. To answer this question, we compared the neural representation of subjective value under risk and ambiguity. fMRI was used to track brain activation while subjects made choices regarding options that varied systematically in the amount of money offered and in either the probability of obtaining that amount or the level of ambiguity around that probability. A common system, consisting of at least the striatum and the medial prefrontal cortex, was found to represent subjective value under both conditions.