Cellular, synaptic and network effects of neuromodulation
All network dynamics emerge from the complex interaction between the intrinsic membrane properties of network neurons and their synaptic connections. Nervous systems contain numerous amines and neuropeptides that function to both modulate the strength ...
Metalearning and neuromodulation
This paper presents a computational theory on the roles of the ascending neuromodulatory systems from the viewpoint that they mediate the global signals that regulate the distributed learning mechanisms in the brain. Based on the review of experimental ...
Dopamine-dependent plasticity of corticostriatal synapses
Knowledge of the effect of dopamine on corticostriatal synaptic plasticity has advanced rapidly over the last 5 years. We consider this new knowledge in relation to three factors proposed earlier to describe the rules for synaptic plasticity in the ...
TD Models of reward predictive responses in dopamine neurons
This article focuses on recent modeling studies of dopamine neuron activity and their influence on behavior. Activity of midbrain dopamine neurons is phasically increased by stimuli that increase the animal's reward expectation and is decreased below ...
Actor-critic models of the basal ganglia: new anatomical and computational perspectives
A large number of computational models of information processing in the basal ganglia have been developed in recent years. Prominent in these are actor-critic models of basal ganglia functioning, which build on the strong resemblance between dopamine ...
Dopamine: generalization and bonuses
In the temporal difference model of primate dopamine neurons, their phasic activity reports a prediction error for future reward. This model is supported by a wealth of experimental data. However, in certain circumstances, the activity of the dopamine ...
The computational role of dopamine D1 receptors in working memory
The prefrontal cortex (PFC) is essential for working memory, which is the ability to transiently hold and manipulate information necessary for generating forthcoming action. PFC neurons actively encode working memory information via sustained firing ...
Dopamine controls fundamental cognitive operations of multi-target spatial working memory
This study addresses computationally how the prefrontal cortical circuit performs operations of multiple items in spatial working memory. The basic idea is that dopamine controls the circuit dynamics for the operations by changing the ratio of the NMDA-...
An integrative theory of the phasic and tonic modes of dopamine modulation in the prefrontal cortex
This paper presents a model of both tonic and phasic dopamine (DA) effects on maintenance of working memory representations in the prefrontal cortex (PFC). The central hypothesis is that DA modulates the efficacy of inputs to prefrontal pyramidal ...
Opponent interactions between serotonin and dopamine
Anatomical and pharmacological evidence suggests that the dorsal raphe serotonin system and the ventral tegmental and substantia nigra dopamine system may act as mutual opponents. In the light of the temporal difference model of the involvement of the ...
Local analysis of behaviour in the adjusting-delay task for assessing choice of delayed reinforcement
The adjusting-delay task introduced by Mazur (Quantitative analyses of behavior: V. The effect of delay and of intervening events on reinforcement value, 1987, pp. 55-73) has been widely used to study choice of delayed reinforcers. This paradigm ...
Neuromodulation of decision and response selection
We present a model for the attentional neuromodulation of decision and selection processes. The model assumes that phasic responses in the brain nucleus Locus Coeruleus modulate, via the transmission of norepinephrine, the synaptic efficiency of neural ...
Control of exploitation-exploration meta-parameter in reinforcement learning
In reinforcement learning (RL), the duality between exploitation and exploration has long been an important issue. This paper presents a new method that controls the balance between exploitation and exploration. Our learning scheme is based on model-...
Neuromodulation, theta rhythm and rat spatial navigation
Cholinergic and GABAergic innervation of the hippocampus plays an important role in human memory function and rat spatial navigation. Drugs which block acetylcholine receptors or enhance GABA receptor activation cause striking impairments in the ...
Cholinergic modulation of sensory representations in the olfactory bulb
We present a computational model of the mammalian olfactory bulb (OB) designed to investigate how cholinergic inputs modulate olfactory sensory representations. The model integrates experimental data derived from diverse physiological studies of ...
Acetylcholine in cortical inference
Acetylcholine (ACh) plays an important role in a wide variety of cognitive tasks, such as perception, selective attention, associative learning, and memory. Extensive experimental and theoretical work in tasks involving learning and memory has suggested ...
Sensory-motor gating and cognitive control by the brainstem cholinergic system
As an essential component of ascending activating systems, cholinergic neurons with diffuse projections are supposed to be involved in the regulation of cognitive processes such as attention, consciousness, learning, and memory. As for the role of ...
On-line learning in changing environments with applications in supervised and unsupervised learning
An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss ...
Neuromodulation and Plasticity in an autonomous robot
In this paper we implement a computational model of a neuromodulatory system in an autonomous robot. The output of the neuromodulatory system acts as a value signal, modulating widely distributed synaptic changes. The model is based on anatomical and ...
