Neural circuits trained with standard reinforcement learning can accumulate probabilistic information during decision making

Journal article

Much experimental evidence suggests that during decision making neural circuits accumulate evidence supporting alternative options. A computational model well describing this accumulation for choices between two options assumes that the brain integrates the log ratios of the likelihoods of the sensory inputs given the two options. Several models have been proposed for how neural circuits can learn these log-likelihood ratios from experience, but all these models introduced novel and specially dedicated synaptic plasticity rules. Here we show that for a certain wide class of tasks, the log-likelihood ratios are approximately linearly proportional to the expected rewards for selecting actions. Therefore, a simple model based on standard reinforcement learning rules is able to estimate the log-likelihood ratios from experience, and on each trial accumulate the log-likelihood ratios associated with presented stimuli while selecting an action. The simulations of the model replicate experimental data on both behaviour and neural activity in tasks requiring accumulation of probabilistic cues. Our results suggest that there is no need for the brain to support dedicated plasticity rules, as the standard mechanisms proposed to describe reinforcement learning can enable the neural circuits to perform efficient probabilistic inference.

Authors: Bogacz, R; Kurzawa, N; Summerfield, C

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Massachusetts Institute of Technology Press
• Journal: Neural Computation
• Volume: 29
• Issue: 2
• Pages: 368-393
• Publication date: 2016-10-01
• Acceptance date: 2016-09-09
• DOI: 10.1162/NECO_a_00917
• ISSN: 0899-7667
• Keywords: decision-making; reinforcement learning; evidence accumulation