I study how people and animals learn from trial and error (and from
rewards and punishments) to make decisions, combining computational,
neural, and behavioral perspectives. I focus on understanding how
subjects cope with computationally demanding decision situations,
notably choice under uncertainty and in tasks (such as mazes or chess)
requiring many decisions to be made sequentially. In engineering, these
are the key problems motivating reinforcement learning and Bayesian
decision theory. I am particularly interested in using these
computational frameworks as a basis for analyzing and understanding
biological decision making. Some ongoing projects include:
that the brain contains multiple, separate decision systems is as
ubiquitous (in psychology, neuroscience, and even economics) as it is
bizarre. For instance, much evidence points to competition between a
reflective or cognitive planning system centered in prefrontal cortex,
and a more stimulus-bound 'habitual' controller associated with dopamine
and the basal ganglia. Such competition has often been implicated in
self-control issues such as dieting or drug addiction. But (as these examples suggest) having multiple solutions to the problem of making
decisions actually compounds the decision problem, by requiring the
brain to choose between the systems. The computational underpinnings and
neural substrates for this sort of arbitration are poorly understood. I
have pursued computational models of multiple decision making systems
and their interactions; armed with such a detailed characterization, we
are beginning to search for the fingerprints of these interactions in
Much evidence has now amassed for
the idea that the neuromodulator dopamine serves as a particular sort of
teaching signal for reinforcement learning in appetitive tasks. This
relatively good characterization can now provide a foothold for
extending this understanding in a number of exciting new directions.
These include computational (e.g., how can this system balance the need
to explore unfamiliar options versus exploit old favorites), behavioral
(how is dopaminergically mediated learning manifest; how is it deficient
in pathologies such as drug addiction or Parkinson's disease), and
neural (what is the contribution of systems that interact with dopamine,
such as serotonin and the prefrontal cortex). One recent example that
crosscuts these categories is the interaction of appetitive and aversive
learning. Psychologists have long suggested that the brain contains
parallel, opponent motivational systems for reward and punishment; the
identification of the former with dopamine allowed us to suggest an
account of serotonin as its opponent for aversive learning. We are presently investigating these ideas with
imaging and pharmacological studies of decision making under reward and
Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P., and Dolan, R.J. (2011, in press) “Model-based influences on humans’ choices and striatal prediction errors.” Neuron.
Cools, R., Nakamura, K., and Daw, N.D., (2011) “Serotonin and dopamine: Unifying affective, activational, and decision functions,” Neuropsychopharmacology 36:98-113.
Gershman, S.J., Pesaran, B., and Daw, N.D. (2009), “Human reinforcement learning subdivides structured action spaces by learning effector-specific values,” Journal of Neuroscience 29:13524-31.
Dayan, P., and Daw, N.D., (2008) “Decision theory, reinforcement learning, and the brain,” Cognitive, Affective, and Behavioral Neuroscience 8:429-453.
Daw, N.D.*, O’Doherty, J.P.*, Dayan, P., Seymour, B., and Dolan, R.J. (2006) “Cortical substrates for exploratory decisions in humans,” Nature 441:876-879.
Courville, A.C.*, Daw, N.D.*, and Touretzky, D.S. (2006) “Bayesian theories of conditioning in a changing world,” Trends in Cognitive Sciences: 10:294-300.
Daw, N.D., Niv, Y., and Dayan, P. (2005) “Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control,” Nature Neuroscience 8:1704-1711.
Daw, N.D., Kakade, S., and Dayan, P. (2002) “Opponent interactions between serotonin and dopamine,” Neural Networks 15:603-616.