Hennig, J. A.; Burrell, M.; Uchida, N. A.; Gershman, S. J.

Animals exposed to pairings of a neutral stimulus with reward acquire a conditioned response to the neutral stimulus. A prominent hypothesis, formalized in the Temporal Difference (TD) learning algorithm, is that animals learn to predict the future reward associated with the neutral stimulus ("value"). Though the TD algorithm does not explicitly specify what drives conditioned responding, a typical assumption is that it reflects the animal's estimate of value. In TD learning, value estimates are updated using reward prediction error (RPE, the discrepancy between observed and predicted reward), and are thought to be signaled by the phasic activity of midbrain dopamine neurons. This hypothesis posits that dopamine's effects on conditioned responding are mediated entirely by its effects on learning. However, recent experimental and theoretical evidence suggests that dopamine may play a more direct role in modulating conditioned responding. We use a combination of data analysis and computational modeling to probe the relationship between dopamine and conditioned responding. Our results suggest that dopamine directly modulates conditioned responding, in addition to its role in learning. These findings can be captured by a model in which dopamine RPE acts both indirectly (via learning) and directly on conditioned responding.