Tonic dopamine, uncertainty and basal ganglia action selection

Abstract To make optimal decisions in uncertain circumstances flexible adaption of behaviour is required; exploring alternatives when the best choice is unknown, exploiting what is known when that is best. Using a detailed computational model of the basal ganglia, we propose that switches between exploratory and exploitative decisions can be mediated by the interaction between tonic dopamine and cortical input to the basal ganglia. We show that a biologically detailed action selection circuit model of the basal ganglia, endowed with dopamine dependant striatal plasticity, can optimally solve the explore-exploit problem, estimating the true underlying state of a noisy Gaussian diffusion process. Critical to the model’s performance was a fluctuating level of tonic dopamine which increased under conditions of uncertainty. With an optimal range of tonic dopamine, explore-exploit decision making was mediated by the effects of tonic dopamine on the precision of the model action selection mechanism. Under conditions of uncertain reward pay-out, the model’s reduced selectivity allowed disinhibition of multiple alternative actions to be explored at random. Conversely, when uncertainly about reward pay-out was low, enhanced selectivity of the action selection circuit was enhanced, facilitating exploitation of the high value choice. When integrated with phasic dopamine dependant influences on cortico-striatal plasticity, the model’s performance was at the level of the Kalman filter which provides an optimal solution for the task. Our model provides an integrative account of the relationship between phasic and tonic dopamine and the action selection function of the basal ganglia and supports the idea that this subcortical neural circuit may have evolved to facilitate decision making in non-stationary reward environments, allowing a number of experimental predictions with relevance to abnormal decision making in neuropsychiatric and neurological disease.