Modelling ADHD Uncorrected Proof

Neuroscience and Biobehavioral Reviews xxx (2016) xxx-xxx
Contents lists available at ScienceDirect
Neuroscience and Biobehavioral Reviews
F
journal homepage: www.elsevier.com
OO
Review article
Modelling ADHD: A review of ADHD theories through their predictions for

computational models of decision-making and reinforcement learning
Sigurd Ziegler a, b, ⁎, Mads L. Pedersen b, c, Athanasia M. Mowinckel b, Guido Biele b, d, ⁎
PR
a
Institute of Clinical Medicine, University of Oslo, PO box 1171, Blindern, 0318 Oslo, Norway
b
Department of Psychology, University of Oslo, PO box 1094, Blindern, 0317 Oslo, Norway
c
The Intervention Centre, Oslo University Hospital Rikshospitalet, PO box 4950, Nydalen, 0424 Oslo, Norway
d
Norwegian Institute of Public Health, PO box 4404, Nydalen, 0403 Oslo, Norway
ARTICLE INFO ABSTRACT
ED
Article history: Attention Deficit Hyperactivity Disorder (ADHD) is characterized by altered Decision-making (DM) and reinforcement
Received 18 September 2015 learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to
Received in revised form 31 August understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic
2016
mechanisms behind ADHD.
Accepted 4 September 2016
Available online xxx
This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as
described by the Drift-Diffusion Model of DM (DDM) and a basic RL model. Empirical studies employing these models
CT
are also reviewed.
Keywords: While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique com-
Attention deficit hyperactivity disorder
bination of predictions. Empirical studies agree with the theories’ assumptions of a lowered DDM drift rate in ADHD,
Basal ganglia model
Boundary separation while findings are less conclusive for boundary separation. The few studies employing RL models, support a lower choice
Computational model sensitivity in ADHD, but not an altered learning rate.
Computational psychiatry The discussion outlines research areas for further theoretical refinement in the ADHD field.
Decision-making
RE
© 2016 Published by Elsevier Ltd.

Dopamine
Dopamine transfer deficit theory
Drift rate
Drift-Diffusion model
Dynamic developmental theory
Learning rate
Moderate brain arousal model
R
Neurobiology
Non-decision time
Noradrenaline
Norepinephrine
CO
Pathogenesis
Prediction error
Reinforcement learning
1. Introduction Willcutt et al. (2008) showed that differences in reward1 discounting
have comparable effect sizes to executive function deficits in children.
Attention Deficit Hyperactivity Disorder (ADHD) is a common But despite the potentially important clinical implications of impaired
neurodevelopmental disorder associated with adverse outcomes in DM, it is still unclear what cognitive and neurobiological mechanisms
academic achievement, occupational status and general wellbeing, and give rise to DM impairments in ADHD (Castellanos and Tannock,
UN
thus also with large health care and societal costs (Biederman and 2002; Cortese, 2012; Hill and Schoener, 1996).
Faraone, 2005; Nigg, 2013). ADHD was originally thought to be a dis- One approach that has proved useful in disentangling the under-
order of childhood and early adolescence (Hill and Schoener, 1996; pinnings of RL and DM in general is computational modelling. Ow-
Nigg, 2013), but current evidence implies persistence into adulthood is ing to its mathematical form, this approach necessitates precise, quan-
common (Biederman et al., 2010). Meta-analyses estimate a cross-na- titatively stated hypotheses of the relationships between psychologi-
tional average prevalence of 5.3% in children (Polanczyk et al., 2014) cal and/or neurobiological processes assumed to be involved in DM
and 3.4% in adults (Fayyad et al., 2007). (Montague et al., 2004; Smith and Ratcliff, 2004). In many cases
ADHD is characterized by deficient executive control — e.g. computational models also utilize more information in the analysis of
age-inappropriate levels of inattention, impulsivity and motoric activ- task performance (e.g. using response time and accuracy, instead of
ity (Willcutt et al., 2005). In addition, deficiencies in decision-mak- analysing them separately), and can thus be more sensitive than tradi-
ing (DM) (Barkley et al., 2001; Frank et al., 2007a; Sonuga-Barke tional measures (Huang-Pollock et al., 2016; Metin et al., 2013). This
and Fairchild, 2012) and reinforcement learning (RL) (Luman et al., could enable the detection of subtle cognitive differences and a more
2010; Maia and Frank, 2011; Nigg, 2013; Sagvolden et al., 2005) fine-grained understanding of the mechanisms underlying cognition
have more recently been described as prominent. A meta-analysis and psychiatric pathologies.
showed that the DM deficits of adults with ADHD have effect sizes Computational modelling is of particular interest for the under-
comparable to attention deficits (Mowinckel et al., 2015). Similarly, standing of neurodevelopmental disorders, as modelling has been suc-
2 Neuroscience and Biobehavioral Reviews xxx (2016) xxx-xxx
agree about the involvement of the brain’s catecholamine systems in 2. Basic neurobiology relevant to ADHD, DM and RL
ADHD, and consequently hypothesize deficits in DM and learning, as
these functions depend on the same neurotransmitters (Doya, 2008; The ADHD theories reviewed herein all attribute the symptoms
Schultz, 2010, 2007a). However, theories differ in their assumptions of ADHD to deficiencies in the DA system, and some partially the
about the specific neurobiological mechanisms underlying ADHD. NA system. These are neurotransmitters known to be involved in DM
F
The aim of this paper is thus to provide a review of the path- and RL. Examples are the role of DA for updating reward expecta-
ogenic mechanisms assumed by the most neurobiologically detailed tions (Schultz, 2010; 2007a), holding information online in working
memory (Arnsten and Pliszka, 2011; Vijayraghavan et al., 2007), ad-
OO
theories of ADHD. Following the computational psychiatry approach,
this is done by analysing current theories through the framework of justing decision thresholds (Bogacz et al., 2010; Green et al., 2012;
two prominent computational models. This approach describes what Lo and Wang, 2006) and in cognitive flexibility (Aarts et al., 2011;
theories directly state or imply about deficiencies in DM, as viewed Arnsten and Pliszka, 2011), or NA in arousal and focused attention
through the Drift-Diffusion Model (DDM) (Ratcliff, 1978; Ratcliff and (Aston-Jones and Cohen, 2005; Berridge and Waterhouse, 2003). The
McKoon, 2008; Smith and Ratcliff, 2004), and RL, as viewed through following is a short review of the anatomical and neurophysiological
a basic prediction error reinforcement learning model (here referred to basis that the ADHD theories build on, as well as their most com-
as PERL) similar to the Q-learning model described by Watkins and monly accepted connections to behaviour.
PR
Dayan (1992). Both models decompose DM and RL into sub-func-
tions that allow understanding deficits that arise from neurobiological 2.1. The DA system
alterations on a more mechanistic level. The modelling framework can
thus clarify the link between the biological and the behavioural level 2.1.1. DA pathways and signalling
implied by the different theories, which enables the use of modelling DA is produced in cells in the midbrain substantia nigra pars
in empirical research to confirm, revise, and/or develop a theoretical compacta (SNc) and the ventral tegmental area (VTA). Three main
understanding of ADHD. dopaminergic pathways project from here (Meyer and Quenzer,
After a brief introduction to the basic neurobiology relevant to 2013): (1) A nigrostriatal pathway projects from SNc to the dorsal
ED
ADHD and the two computational models, this paper gives a review striatum (caudate nucleus and putamen); (2) a mesolimbic pathway
of four prominent ADHD theories, including a brief assessment their projects from the VTA to limbic system structures like the ventral
main neurobiological assumptions. Building on this, the paper then de- striatum (nucleus accumbens), hippocampus and amygdala; and (3) a
scribes the predictions for DDM and PERL parameters entailed by the mesocortical pathway projects from the VTA to the cerebral cortex,
theories. Predictions are then compared to current empirical findings especially medial prefrontal areas.
from studies applying relevant computational models, as well as com- DA is a neuromodulator that exerts its actions through excita-
CT
pared between theories. Building on this, the review suggests experi- tory D1-like receptors (D1 and D5) and inhibitory D2-like receptors
ments that could discriminate between theories. (D2, D3, D4), and is released in the striatum and the prefrontal cortex
(Arnsten et al., 2015; Berridge and Waterhouse, 2003; Robbins and
Arnsten, 2009; Schultz, 2007a). Reuptake is the most important reg-
ulator of synaptic DA levels, and in the striatum this happens mainly
through the dopamine transporter (DAT) (Giros et al., 1996). DA in
R RE
CO
UN
Neuroscience and Biobehavioral Reviews xxx (2016) xxx-xxx 3
the prefrontal cortex is mainly removed through reuptake by the NA stimulation has the opposite effect on SNR, thus enabling flexible
transporter and breakdown by catechol-O-methyltransferase (COMT), processing and set-shifting (Durstewitz and Seamans, 2008; Kreitzer,
since DAT is minimally expressed outside of the striatum (Arnsten 2009). Similarly, striatal DA release enhances neural activity result-
and Pliszka, 2011; Berridge and Devilbiss, 2011; Käenmäki et al., ing from strong, presumably task-relevant cortical input, while also
2010). Stimulant medications likely exert their effect in ADHD suppressing weaker spontaneous or task-irrelevant firing (Gerfen and
Surmeier, 2011).
F
through inhibiting reuptake transporters of DA and NA (Berridge and
Devilbiss, 2011). DA also helps the brain to separate between relevant stimuli and
“noise” on a longer time scale, as striatal DA contributes to long-term
OO
2.1.2. Tonic and phasic DA potentiation and long-term depression. The concurrence of cortical
Accounts of DA-signalling often discern between tonic and pha- presynaptic and striatal postsynaptic firing with correctly timed pha-
sic DA levels. Tonic levels refer to the slowly varying concentration sic DA leads to long-term potentiation in the involved synapse, while
of DA in the extracellular fluid surrounding dopaminergic terminals. DA depletion can block long-term potentiation (Tripp and Wickens,
Tonic levels are derived from spontaneous neural firing and can stim- 2012). Thus, reward-related DA release increase the connectivity of
ulate extrasynaptic receptors, supporting the view of DA working also pathways involved in rewarded, goal-relevant behaviour.
through slow volume transmission (Rice and Cragg, 2008). Levels
of tonic DA may rise upon sustained DA neuron firing, presynaptic
PR
2.2.1. Corticostriatal neural circuitry
glutamatergic stimulation from cortical afferents or blocking of the Several theories hold the corticostriatal system as central in the
dopamine reuptake (Grace, 2000; Grace et al., 2007). Tonic DA is not pathogenesis of ADHD due to its rich dopaminergic innervation and
removed primarily by reuptake, but likely by enzymatic breakdown, involvement in reward-processes as well as in sustained attention and
as opposed to the synaptic phasic DA (Grace et al., 2007), and it is a working memory.
potent inhibitor of spike-dependent phasic DA release through its ef- The most common cells in the striatum are two subtypes of
fects on presynaptic D2 autoreceptors (Grace, 1995, 1991). GABAergic medium spiny projection neurons (MSNs) with antag-
Phasic release designates the brisk, mainly intrasynaptic spikes of onising effects on the circuit level and different reactions to DA
ED
DA-concentration that occur upon event-related firing of midbrain (Brodal, 2010; Gerfen and Surmeier, 2011). One subtype expresses se-
DA-neurons. During phasic release, DA peaks at high concentrations lectively D1 receptors, and gives rise to the so-called direct pathway to
which possibly affect more the low-affinity D1-receptors (Dreyer et the thalamus, while the other subtype expresses selectively D2 recep-
al., 2010). Phasic dips occur when DA neurons shut down their ac- tors and project to the thalamus through an indirect pathway (Brodal,
tivity and rapid DA reuptake ensures a reduction in synaptic DA 2010; Gerfen and Surmeier, 2011).
(Schultz, 2007a). A recent DA transmission model integrating experi- Striatal MSNs are part of a cortical feedback loop and receive
mentally derived parameters show that phasic dips reduce receptor oc- excitatory glutamatergic input from the frontal cortex (Gerfen and
CT
cupancy in both D1 and D2-receptors (Dreyer et al., 2010). Surmeier, 2011). The end result of D1 MSN-firing is disinhibition of
Tonic and phasic DA have been linked to different behavioural the cortical region from where the cells received the original input,
functions (Goto et al., 2007; Schultz, 2007a; Tsai et al., 2009). Pha- which strengthens the cortical activity and facilitates execution of mo-
sic DA release in the striatum is important for evaluation and RL tor or cognitive processes. Conversely, firing of the D2 MSNs results
(Doya, 2008; Glimcher, 2011; Niv and Schoenbaum, 2008; Schultz, in inhibition of the cortical signal source, although recent findings sug-
2010, 2007a), but has also been linked to incentive salience (Berridge, gest D2 MSNs are also involved in the early phase of the onset of
RE
2012). The view of DA as a carrier of the brain’s reward prediction movements (Cui et al., 2013). The crucial point is that DA has oppo-
error signal forms the basis of this article, and this is elaborated in site effects on the activity of D1 MSNs and D2 MSNs because of their
section 3.2. The slowly fluctuating tonic DA may have a more mod- different receptor subtypes, and increases and decreases their firing re-
ulatory, as opposed to an information-carrying, function, and levels spectively, thus giving more D1 driven disinhibition and less D2 driven
are affected by e.g. uncertainty, aversive stimuli, novelty, reward and inhibition of the cortex (Brodal, 2010; Gerfen and Surmeier, 2011). It
movement (Schultz, 2007a). has also been hypothesised that the direct and indirect pathway are re-
R
DA-release on different time scales might also have divergent roles sponsible for learning the positive and negative value of states (Maia
in the cortex and the striatum (Bilder et al., 2004). In the prefrontal and Frank, 2011).
cortex, tonic and phasic DA is hypothesised to have different roles in
maintenance and updating of WM, respectively (Cohen et al., 2002).
CO
Striatal phasic and tonic DA can also selectively regulate input control 2.3. The NA system
of the ventral striatum by the limbic system and the mPFC, respec-
tively, through different effects on D1 and D2-receptors (Grace et al., The neuromodulator NA is synthesized from DA by the enzyme
2007). Thus, tonic and phasic DA could modulate the balance between dopamine-β-hydroxylase (Berridge and Waterhouse, 2003). NA neu-
learning and exploitation of reward behaviour on one side, and strat- rons have their cell bodies in the pons and the upper medulla ob-
egy-switching on the other (Goto et al., 2007). longata, and are particularly densely clustered in the pontine locus
UN
coeruleus (Berridge and Waterhouse, 2003). Axons from noradrener-

2.2. DA and signal-to-noise ratio gic cells project diffusely to most of the forebrain and also to the cere-
bellum and spinal cord, while the basal ganglia are sparsely innervated
D1-stimulation differentiates the output between neurons respond- by NA (Berridge and Waterhouse, 2003).
ing to strong input and those receiving random or irrelevant input, Receptors for NA are divided into α and β-type receptors. β1 and
thus increasing signal-to-noise ratio (SNR) (Durstewitz and Seamans, work similarly to D1-type DA receptors in the brain, while α2-re-
β2
2008). In the cortex, increased SNR is achieved through stabilis- ceptors work analogously to D2-receptors and can similarly be found
ing working memory representations in dorsolateral prefrontal parts as presynaptic inhibitory autoreceptors (Meyer and Quenzer, 2005 p.
(Arnsten, 2013; Vijayraghavan et al., 2007) or value representations 134). NA signalling is terminated through reuptake by the presynap-
in the orbitofrontal cortex (Maia and Frank, 2011; Schultz, 2000). D2- tic NA transporter (Berridge and Waterhouse, 2003), which impor
tantly is also blocked by stimulant medication (Berridge and cision boundary in favour of an option is reached, upon which a re-
Devilbiss, 2011). sponse is executed. Consider deciding between an apple and a pear as
NA is known to increase arousal, vigilance and attention. Evidence an after lunch snack. This decision process is represented in Fig. 1 by
from animal research connects tonic levels of NA to arousal, while a sample path with a starting point z. The difference in evidence − in
bursts of phasic NA are associated with focused attention and en- this case the difference in subjective value between the apple and the
hanced responses to task-relevant stimuli (Berridge and Waterhouse, pear − is continuously accumulated until a boundary for one of the two
F
2003; Pliszka, 2005; Robbins and Arnsten, 2009). Phasic discharge alternatives (e.g. upper or lower boundary representing “apple” and
is sensitive to tonic levels, and both too high and too low tonic NA “pear”, respectively) is reached. As can be seen, the process is affected
OO
can impair the phasic response, and thus focused attention (Berridge by random noise.
and Waterhouse, 2003). Low tonic levels are observed during drowsi- According to the DDM, accuracy and response time distributions
ness and other non-alert states, while high tonic levels are tied to depend on a number of decision variables or free parameters. One im-
the high arousal occurring e.g. under appetitive or threatening condi- portant decision variable that reflects task difficulty or information
tions, where scanning of the environment and a widened scope of at- processing ability (which can again depend on attention) is the drift
tention can be adaptive (Aston-Jones and Cohen, 2005; Berridge and rate (v). This is the average speed with which the evidence difference
Waterhouse, 2003). signal, and thus the decision process, approaches a response boundary.
NA is assumed to facilitate response execution through work- High drift rates lead to faster and more accurate decisions.
PR
ing as a selective amplifier and is modelled as affecting the gain A second important variable is the boundary separation, which
(i.e. the slope) of the activation function of cortical premotor units describes the amount of accumulated evidence needed until a deci-
(Aston-Jones and Cohen, 2005). Strong, task-related activity (which sion is made. The boundary separation determines the speed-accuracy
may reflect earlier reinforcement) normally elicits phasic NA release trade-off (Bogacz et al., 2010), where low boundary separations leads
from the locus coeruleus, which again “sharpens” relevant representa- to faster, but less accurate, choices than decisions made with a larger
tions compared to competing activity. The result is a positive feedback boundary separation. Reduced accuracy is then due to random neural
loop. noise being more likely to direct the evidence signal to the wrong
ED
The inverted U-shaped relationship between performance on some threshold and thus triggering an erroneous decision when the bound-
tasks and neurotransmitter levels has been observed with both with ary separation is low.
NA and DA, and is tied to e.g. the trade-off between exploitation The position of the starting point z reflects the extent to which one
and exploration required for optimal task performance (Arnsten and decision alternative is preferred over the other, independently of evi-
Pliszka, 2011; Aston-Jones and Cohen, 2005; Cools and D'Esposito, dence for the current decision. Additionally, the DDM can distinguish
2011; Goldman-Rakic et al., 2000). This also implies that the benefi- effects of stimulus encoding and motor execution — which are cap-
cial effect of medication affecting these systems can depend on base- tured by a non-decision-time parameter (Ter) — on RTs from effects
CT
line transmitter levels as well as the requirements of the task, e.g. if the of drift rate and boundary separation. It is also possible to model the
task requires cognitive/attentive flexibility or stability. The NA sys- inter-trial variability of the drift rate, bias and non-decision time as
tem appears to have overlapping functions with the DA system in reg- free parameters.
ulating vigilance and attention, although it seems to react to salient By estimating decision variables for different conditions and
stimuli in general without giving prominence to rewards (Berridge and groups, the DDM can be used to uncover sub-mechanisms of DM
Waterhouse, 2003). that contribute to the observed differences in accuracy and response
RE
time (RT). Reduced accuracy can be caused either by fast respond-

3. Computational models of decision making and learning ing, as modelled by a smaller boundary separation, or by a slower ev-
idence accumulation (of external origin, as due to low stimulus qual-
The interest in using computational models for bridging human ity, or of internal origin, as through inattention or other causes of
neurobiology to behaviour is increasing (Barkley et al., 2001; Bogacz,
2007; Frank et al., 2007a; Mulder et al., 2014; Smith and Ratcliff,
R
2004). This is largely because models provide mechanistic explana-

tions of catecholamine function in RL and DM (Bogacz et al., 2010;
Frank et al., 2007a, 2004; Green et al., 2012; Krugel et al., 2009; Lo
CO
and Wang, 2006; Suri and Schultz, 2001), and thus may provide more
detailed insight to cognition and learning in psychiatric illness and
dysfunction (Maia and Frank, 2011; Montague et al., 2012). The fol-
lowing sections present the two computational models that are used in
this paper as a framework for analysing ADHD theories.
3.1. Sequential sampling models of decision-making

UN
Sequential sampling models are computational models used to

model the cognitive and neurobiological processes underlying simple
Fig. 1. Main features of the Drift-Diffusion Model. Accumulation of evidence begins at
forced-choice decisions, typically in trials lasting up to a few seconds
a starting point (z). Evidence is represented by sample paths with added Gaussian noise
(Bogacz, 2007). One popular sequential sampling model is the Drift and is gathered until a decision boundary is reached (upper or lower) and a response
Diffusion Model (Ratcliff, 1978; Ratcliff and McKoon, 2008; Smith is made. Paths with different average (opaque = high, translucent = low) drift rates (v)
and Ratcliff, 2004). The DDM assumes that the decision process in- are depicted as paths with different inclinations from z to the boundaries. The distance
volves continuously sampling evidence in favour of two decision al- between the decision boundaries represents the boundary separation (a). Each path rep-
resents one modelled decision trial, and the distributions at each boundary are the dis-
ternatives, and accumulating an evidence difference signal until a de tributions of the response times of the two choices over multiple trials. Figure adapted
from Wiecki et al. (2013), reprinted with permission.
slowed information processing), as modelled through lower drift rate. instrumental learning. This surprise is coded by a prediction error (PE)
For example, the lower accuracy levels often seen in ADHD could be signal, which reflects the difference between predicted and obtained
explained both by tighter decision boundaries — indicating impulsiv- reward. The PE signal is sometimes also called an updating signal, be-
ity — or a lower drift rate — indicating slower information process- cause past expectations are updated based on the PE signal in order to
ing, whereas faster RTs could result from both tighter boundaries and/ generate corrected reward expectations. This basic idea — updating of
F
or a higher drift rate. expectations based on PEs — is the core of modern RL models, in-
Importantly, the DDM does not only successfully describe deci- cluding the popular temporal difference (TD) learning model (Sutton
sions in healthy and clinical populations (Pe et al., 2013; Smith and and Barto, 1998) and the Q-learning model (Watkins and Dayan,
OO
Ratcliff, 2004; White et al., 2010), it is also consistent with neu- 1992), or the Expectancy Valence model frequently used to under-
robiological mechanisms of DM uncovered with neurophysiological stand decision making deficits in psychiatric disorders (Busemeyer
and neuroimaging experiments (Ratcliff, 1978; Ratcliff et al., 2003; and Stout, 2002).
Ratcliff and McKoon, 2008; Smith and Ratcliff, 2004; Usher and A basic PERL model models instrumental learning as follows
McClelland, 2001). The core results are the identification of “accu- (Watkins and Dayan, 1992; following Rescorla and Wagner, 1972):
mulator” neurons, e.g. in the intraparietal sulcus, that receive input The reward expectation (Q) for a given trial (t) is updated by the PE
from lower levels (often perceptual neurons, depending on task), and (δ), which is the difference between observed and expected reward on
PR
accumulate evidence in favour of decision alternatives until a thresh- the last trial:
old is reached (Basten et al., 2010; Gold and Shadlen, 2007; Kim and
Shadlen, 1999; Shadlen and Newsome, 2001). Further research has
implied a role of dopaminergic projections to the basal ganglia for
the adjustment of decision thresholds (Bogacz et al., 2010; Gold and
Shadlen, 2007; Green et al., 2012; Lo and Wang, 2006). More gen- The PE is weighted with a learning rate parameter α so that larger
erally, cortical DA is thought to modulate the SNR of cortical repre- learning rates close to 1 lead to fast adaptation of reward expectations:
sentation and thus potentially influence the drift rate (Durstewitz and
ED
Seamans, 2008; Li et al., 2001).
A key aspect of the DDM is the assumption that its main para-
meters modulating response time also modulate accuracy. Hence, re-
The learning rate α is a free parameter that can vary between indi-
duced accuracy in DM tasks can be caused either by fast responding,
as modelled through a smaller boundary separation, or by a slower ev- viduals or over time within a learning task. High learning rates result
idence accumulation, modelled through a lower drift rate. A low drift in fast learning from recent experience, but forgetting of past events
rate can be of external origin, as due to low stimulus quality, or of in- will also be fast. Small learning rates near 0 lead to slow adaptation,
CT
ternal origin, as through inattention or other causes of reduced infor- but also less influence from random variations in feedback.
mation processing speed. In instrumental learning the decision maker has to choose between
Assumptions about DA and/or NA availability in ADHD theories different options. A choice rule models with which probability
can also inform the theory predictions for the DDM and PERL mod- a decision maker chooses option i among a total of n alternatives with
els. Generally, low levels of cortical DA and/or NA, which are often different expected rewards in a given trial (t):
hypothesized for ADHD, are assumed to lead to a lower SNR of cor-
RE
tical firing (Aston-Jones and Cohen, 2005; Durstewitz and Seamans,

2002), which should in turn map to a lower drift rate. Low levels of
subcortical DA (i.e. in the basal ganglia) could similarly affect sys-
tem level SNR through the striatal filtering of cortical inputs (Nicola et
al., 2004) and/or selective updating of task-relevant information into This function is often referred to as a softmax function (Luce,
working memory (WM) (O'Reilly and Frank, 2006). Finally, DA lev- 1959). The choice sensitivity parameter describes two things: (a)
R
els could have an effect on how ADHD patients trade of speed and sensitivity to expected rewards, and (b) exploration-exploitation
accuracy (Mulder et al., 2010), as the cortical and subcortical regions trade-off. The choice function could also be replaced by the DDM, in
presumably involved in the speed-accuracy trade-off are modulated which the accumulated evidence represents the perceived difference in
CO
by DA (Bogacz, 2007; Forstmann et al., 2008; Mulder et al., 2014). expected payoffs (Pedersen et al., in revision).
Although low DA is assumed to reduce the ability to optimally trade The finding that the reward PE is signalled by midbrain dopamin-
off speed and accuracy, the association between the speed-accuracy ergic neurons (Montague et al., 1996) has fuelled a strong interest in
trade-off and DA is disputed (Winkel et al., 2012). Thus, no predic- RL models. Schultz et al. (Schultz, 2007b; Schultz et al., 1992) col-
tions for boundary separation will be derived directly in this review lected single cell recordings from monkeys, and observed that early
from a theory’s assumptions of DA alterations, unless the theory also in learning DA neurons fired with presentation of the unconditioned
makes additional assumptions that clearly should affect the speed-ac- stimulus. As the association was learned, this firing propagated back-
UN
curacy trade-off. wards to the predicting conditioned stimulus. This is exactly as ex-
pected of a positive PE signal within in the most common compu-
3.2. Reinforcement learning tational RL frameworks. In addition, when the conditioned stimulus
did not appear, a dip in DA firing followed, as would be expected
RL models originated as an attempt to explain instrumental and of a negative PE signal (Schultz, 2007b). Further experiments sup-
associative learning (R. R. Bush and Mosteller, 1951; Rescorla and port the view that DA neurons code the neural reward PE, also in the
Wagner, 1972). Central to modern theories of RL is that learning oc- context of instrumental learning. For instance, the amount of reward
curs when events violate expectations, such as when an unexpected correlates with the firing rate of midbrain DA neurons (Tobler et al.,
reward occurs in the context of classical conditioning or when a 2005), activity of DA neurons are proportional to the probability of
larger or smaller than expected reward is obtained in the context of
receiving reward (Fiorillo, 2003), and the temporal delay of reward 4. Theories of ADHD and computational modelling
correlates negatively with the DA firing at the conditioned stimulus
predicting reward delivery (Roesch et al., 2007). Human functional 4.1. Introduction and approach
magnetic resonance imaging (fMRI) experiments have also identi-
fied brain activity consistent with a dopaminergic PE (Garrison et The next section presents four neurobiological theories of ADHD.
al., 2013; O'Doherty et al., 2004), and correlations between PE and These theories were chosen because they are well-established and
F
brain activity in striatum has been shown to predict instrumental learn- offer detailed mechanistic explanations of ADHD through neural
ing success in humans (Schonberg et al., 2007). More direct evidence processes also known to be involved in RL and DM. After a general
OO
for the role of DA in human learning comes from pharmacological description of each theory, including their neurobiological assump-
studies showing that DA agonists and antagonists lead to improved tions and how they explain important ADHD deficiencies, this article
and reduced learning respectively (Pessiglione et al., 2006), and that briefly discusses the evidence basis of each theory. Next, the article
DA-precursor medication improves learning from rewards in Parkin- presents the predictions implied by the theory for simple decisions,
son’s patients (Frank et al., 2004; Graef et al., 2010). There is also evi- as modelled by the DDM, and for RL, as modelled by a basic PERL
dence linking activity in cortical orbitofrontal and motor regions to the model. As some ADHD theories emphasise developmental contribu-
expected reward of actions (Cohen and Frank, 2009; Maia and Frank, tions to ADHD symptoms, both immediate effects resulting directly
2011; Schultz, 2000). from deficient neurobiological processes and effects that develop over
PR
The PERL model assumes that learning relies on a PE signal re- time secondarily to altered learning will be described. Effects that
flecting the difference between expected and obtained reward. Speed are likely to be changed by an acute increase (or decrease) of cate-
of learning—i.e. of updating of reward expectations—depends on the cholaminergic stimulation will be referred to as direct effects. When
modelling level on the learning rate, which determines how strongly effects are assumed to develop over longer time (weeks to months or
the PE influences the updated expectation. On the neurobiological more), through learning or unlearning of behavioural/cognitive ten-
level, speed of learning is assumed to depend on the strength of the dencies or strategies, this will be referred to as indirect or developmen-
phasic dopaminergic PE signal (Glimcher, 2011). DA responses to tal effects. Such learning might again be secondary to catecholamine
ED
negative PEs or worse-than-expected outcomes are somewhat more alterations. The assumption of indirect, causally more distant effects
complex, but the magnitude and potentially the duration of dips in of altered brain processes are in line with the view of ADHD as a neu-
tonic levels of DA is related to reduction of reward expectations (Niv rodevelopmental disorder (e.g. Sagvolden et al., 2005; Sonuga-Barke,
and Schoenbaum, 2008; Schultz, 2013; 2007b). Given that different 2003). The effects of ADHD on DDM and PERL parameters predicted
ADHD theories make diverging assumptions about the effect of de- by the different theories are summarized in Table 1.
ficient DA modulation on positive and negative PE signals, it makes
sense to conceptualize a model with separate learning rates for pos- 4.2. The basal ganglia model
CT
itive and negative PEs. PERL models commonly use a choice sensi-
tivity parameter to model the strength of the preference for the option 4.2.1. Biological presumptions
with the highest expected reward. If the choice sensitivity parameter is The Basal Ganglia model (BGM) (Frank et al., 2007a) assumes
low, this can reflect a preference for exploration, impulsive choosing low tonic DA, particularly in the striatum, in ADHD. The assump-
and/or low reward motivation, possibly linked to low corticostriatal tion is based the general assumption of a DA signalling deficiency
DA/NA (Durstewitz and Seamans, 2008; Hauser et al., 2016; 2014). in ADHD (e.g. Biederman and Faraone, 2002; Krause et al., 2000;
RE
There is some overlap between the PERL choice rule and the DDM, Sagvolden et al., 2005; Solanto, 2002), as well as evidence showing
as low reward motivation will be picked up as a lower DDM drift rate, that stimulants increase tonic DA in healthy controls (Volkow et al.,
while impulsive choosing will affect boundary separation. 2001).
Table 1
Summary of model parameter predictions. The table summarizes predictions for difference in model parameter values in ADHD relative to typically developing controls. Note that
R
even though theories often make similar predictions for both the DDM and the simple PERL model, they can differ in the conditions required for observing the effect. Important
conditions required for observing the directional change of a parameter are noted in parenthesis; please consult the main text for details. White cells indicate predictions are based on
direct effects of ADHD; grey cells indicate a contributing developmental effect, indirectly through deficient learning. Learning rates are for positive prediction errors (rewards, PE+)
and negative prediction errors (worse-than-expected outcomes, PE–).
CO
Change in model parameters relative to typically developing controls
Drift-Diffusion Model PERL model
Drift rate Boundary separation Learning rate PE+ Learning rate PE– Choice sensitivity
Basal Ganglia ↓/↑ ↓ ↓ ↑ ↓

model (approach/avoidance)
UN
Dynamic ↓ ↓ ↓ ↓ ↓
Developmental (difference increasing during course (difference increasing during course (especially when (difference increasing during
Theory of task; ameliorated by frequent of task; ameliorated by frequent losses are large) course of task; ameliorated by
reinforcers) reinforcers) frequent reinforcers)
Dopamine ↓ ↓/= ↓/=/↑ ↑/= ↓/=
Transfer (depending on error cost) (delayed/immediate (omission of (depending on error cost)
Deficit theory rewards/late in delayed/immediate
trial) reward)
Moderate Brain ↓/= ↓/= ↑/=/↓ ↑/=/↓ ↓/=
Arousal model (low or high/moderate arousal) (low or high/moderate arousal) (low/moderate/high (low/moderate/high (low and high/moderate arousal)
arousal) arousal)
Phasic DA in striatum is also thought to be low in ADHD. This low, the need to integrate the net signal over longer time leads to slow
assumption is supported by animal work showing stimulants also in- RTs (Frank et al., 2007b,a).
crease intrasynaptic DA (Ruskin et al., 2001; Schiffer et al., 2006), Finally, according to the BGM, DA-related Go-learning deficien-
as well as by the Dynamic Developmental Theory of Sagvolden et al. cies should be correlated with each other, but not with the deficiencies
(2005). assumed by the model to be caused by inadequate NA transmission
With respect to NA, the BGM assumes elevated tonic NA, leading (e.g. erratic choice behaviour and variable RTs). This prediction has
F
to reduced phasic NA, in ADHD. This is consistent with findings that also been supported by the authors’ findings (Frank et al., 2007a).
cortical NA in general affects RT variability (Aston-Jones and Cohen,
OO
2005; Frank et al., 2007b), of heightened PFC NA in rat ADHD mod- 4.2.3. Limitations and discussion of the theory
els (e.g. Russell and Wiggins, 2000) and that urinary NA metabolites The BGM is inconsistent with the tonic/phasic model of DA
correlate with RT-variability in ADHD (Llorente et al., 2006). Clinical (Grace, 2000, 1991) by assuming low levels of both tonic and phasic
effects of selective NA transporter-blockers also supports this claim DA in ADHD. The authors support this assumption with findings of a
(Chamberlain et al., 2006; Overtoom et al., 2003). DA and NA defi- general DA signalling deficiency and indirect evidence from pharma-
ciencies are assumed to be independent in ADHD (Frank et al., 2007a; cological or animal studies (Biederman and Faraone, 2002; Krause et
Llorente et al., 2006). al., 2000; Ruskin et al., 2001; Sagvolden et al., 2005; Schiffer et al.,
PR
2006; Solanto, 2002). However, the study by Ruskin et al. (2001) was
4.2.2. Theory description on healthy animals, and Schiffer et al. (2006) applied stimulant doses
Frank et al. (Frank et al., 2007b,a; Frank and O'Reilly, 2006) use above normal clinical doses (Faraone, 2010; Fredriksen et al., 2013;
a neurocomputational model of frontostriatal DA and NA to under- Sikstrom and Söderlund, 2007). Molecular imaging (PET, SPECT)
stand these transmitters’ contribution to deficits in instrumental learn- has given inconsistent results about direction of change of DA metab-
ing and working memory in ADHD. Their BGM builds on the canon- olism, receptor availability, stimulant response and DAT activity in
ical framework of corticostriatal circuitry described in section 2.1. ADHD (del Campo et al., 2011; Volkow et al., 2009; Zimmer, 2009).
D1-MSNs are referred to in the BGM as Go-cells and D2-MSNs But a recent PET-study directly estimating tonic vs. phasic DA in
ED
as NoGo-cells. Learning actions from reward is called Go-learning, ADHD found reduced tonic DA and activity-increased phasic DA in
while learning to avoid worse-than-expected outcomes (punishment or the right caudate nucleus in ADHD during a flanker task (Badgaiyan
reward omissions) is called NoGo-learning. In this context, the term et al., 2015). As in all PET-studies, conclusions are drawn from mea-
“actions” encompasses motoric output, working memory updating and surements with a substantially lower temporal resolution than the
decision-making (Frank et al., 2007a). sub-second dynamics of phasic DA-release. Still, if this finding is cor-
According to the BGM, ADHD is associated mainly with impair- roborated by further studies, the BGM may need to be adapted to this
ments in Go-learning caused by weak DA signals after positive pre- finding.
CT
diction errors (Frank et al., 2007a; Sagvolden et al., 2005). Conse- There is less direct evidence on NA in ADHD, partially because
quently, the model posits that only deficits in Go-learning should be appropriate ligands for molecular imaging have not been available for
alleviated by central stimulants, as these are assumed to increase pha- long (del Campo et al., 2011). Inattention measures correlate with uri-
sic DA in the striatum (del Campo et al., 2011; Wilens, 2008). Low nary NA metabolites, supporting a connection between elevated NA
tonic baseline DA levels should also give better NoGo-learning ac- and inattention, although correlations are found to be low to moderate
cording to the model, as is seen in patients with Parkinson’s disease (Llorente et al., 2006).
RE
(PD) (Frank et al., 2004). This implies a worsening of NoGo learning Another issue is the lack of empirical support of the model’s pre-
with stimulants, since these attenuate the phasic DA dips associated diction of increased NoGo-learning in ADHD, as is found in the hy-
with worse-than-expected outcomes. This prediction was not empiri- podopaminergic PD (Frank et al., 2004). However, PD is caused by
cally supported, however, possibly due to a floor effect of NoGo learn- cell death in DA cell clusters, not just altered signalling. This im-
ing in the study (Frank et al., 2007a). plies a greater reduction in DA levels that likely result in large be-
The BGM model also posits that Go-signalling lowers the thresh- havioural alterations that are more easily detected in studies. Frank
R
old for updating working memory contents (Frank et al., 2007a). A et al. (2007a) also cite findings of D2-receptor hypersensitivity in PD
striatal DA-deficiency thus implies difficulties in selective gating of that might increase NoGo-signalling in PD, but not in ADHD (Rinne
relevant stimuli into working memory. This means that people with et al., 1990). It should however be mentioned that hypersensitivity is
CO
ADHD perform worse than TDCs on tasks depending on selective also implied for D1-receptors in PD (Gerfen et al., 2002), without this
working memory gating, e.g. when distractors are present, but also seeming to increase Go-learning, which should then follow from the
should perform better on tasks where information in working memory assumptions of the BGM (Frank et al., 2004).
should be ignored not to interfere with immediately presented stim-
uli, a prediction that is supported by the authors’ study (Frank et al., 4.2.4. Predictions for drift-diffusion model parameters
2007a).
Moreover, the variability of RTs often seen in ADHD is accounted 4.2.4.1. Drift rate
UN
for in the BGM by an independent NA component (Frank et al., Low tonic and phasic DA in ADHD should lead to decreased
2007a). In the BGM, ADHD can also be associated with increased Go-signalling and likely some increase in NoGo-signalling, following
tonic and reduced phasic NA, which gives an inflexible, activity-in- the BGM (Frank et al., 2007a). This should manifest as an insensitivity
dependent amplification of cortical activity. This results in less sig- to gains and increased loss aversion. Thus, people with ADHD should
nal differentiation and thus increased distractibility. When response be faster and more accurate on avoiding negative outcomes. When a
conflict in the brain is low, this increased tonic NA is hypothesised task requires an active response to avoid a loss, e.g. a button press
to give a high gain in premotor units and normal RTs (Frank et to reject an option with negative value, people with ADHD should
al., 2007b, 2007a). However, if conflict is high, and differentiation show increased drift rate towards this choice. If loss can be avoided
between the competing signals fed forward to the basal ganglia is through choosing a gain, then the drift rate towards the positive op
tion should increase. When choosing between gains, drift rate should
be lower in ADHD.
Reduced phasic DA also decreases the gating of task-related stim-
uli into WM, thus reducing performance in tasks that tap into WM.
Together with reduced cortical signal differentiation due to deficient
phasic NA, this should give overall reductions and more variable drift
F
rates compared to controls, especially for high-conflict or difficult/am-
biguous choices, or when distractors are present (Frank et al., 2007b).
OO
Still, people with ADHD should exhibit higher drift rates for choices
avoiding negative outcomes compared to choices approaching posi-
tive outcomes.
4.2.4.2. Boundary separation

Frank et al. (2006, 2007b) elaborate on a possible role for the sub-
thalamic nucleus (STN) in providing a global “hold you horses” signal
for preventing premature responding. Based on earlier empirical work Fig. 2. Basal ganglia pathways. The direct pathway starts with D1-neurons (Go-cells)
PR
(Arcos et al., 2003; Aron and Poldrack, 2006; Orieux et al., 2002), the that project directly to the internal globus pallidus (GPi), which in turn projects back to
the cortex via the thalamus. The indirect pathway starts with D2-neurons (NoGo-cells)
STN is modelled as effectuating inhibitory actions that are dynami- that project via the external globus pallidus (GPe) and the subthalamic nucleus (STN) to
cally modulated by the degree of decision conflict. The STN is also the GPi, thalamus and cortex. Dopamine (DA) from the substantia nigra pars compacta
assumed to be affected by NA, either directly, or via frontal cortical (SNc) or the ventral tegmental area activates Go-cells and inhibits NoGo-cells. Figure
areas (Frank et al., 2007b), thus implying that the low phasic NA ac- from Frank (2006), reprinted with permission. Copyright © [2006], Elsevier.
tivity in ADHD should lead to a lowered boundary separation. One
important addendum is that since the boundary separation is sensitive Low tonic DA leads to a floor-effect that limits phasic DA reduc-
tions in response to worse-than-expected outcomes in ADHD. This
ED
to decision conflict, one should generally find higher boundary sepa-
ration for difficult decisions (Frank et al., 2007b), but presumably still hypothesis is deduced from the general association between reward
lower overall in ADHD than in TDCs. omissions and phasic depressions of DA-neuron firing (Schultz, 2002)
and reduced extinction in ADHD (Sagvolden et al., 1998) and in the
4.2.5. Predictions for PERL parameters SHR (Johansen and Sagvolden, 2004).
4.2.5.1. Learning rate 4.3.2. Theory description

CT
The BGM clearly predicts a reduced learning rate for positive PEs The DDT (Sagvolden et al., 2005) is to a large extent based on
or Go-learning due to low phasic DA levels (Frank et al., 2007a). In research with the SHR animal model of ADHD (see e.g. Sagvolden,
general, the model also implies an increased learning rate for negative 2000). According to the theory, impaired RL is the main pathogenic
PEs or NoGo trials when DA is low, although this was not found em- factor in ADHD. The neural mechanism behind this impairment is low
pirically in one study (Frank et al., 2007a). tonic DA that leads to a low phasic DA signal in the brain’s corti-
costriatal loops (Fig. 4A), most importantly in the limbic loop (Grace,
RE
4.2.5.2. Choice sensitivity parameter 2002; Russell et al., 1995; Solanto et al., 2001b). This results in a
High phasic NA in the BGM sharpens cortical representations and reduced positive reward prediction error signal in the ventral stria-
reduces variability of RTs, while the intermediately elevated tonic and tum, which gives a shorter time window for relating reward-predictive
reduced phasic NA of ADHD enables representation of multiple op- stimuli or actions to reinforcements. This is conceptualised in the DDT
tions and favours exploratory behaviour (Frank et al., 2007b). There- as a more steeply decaying delay-of-reinforcement gradient (Catania
fore, the choice sensitivity parameter should be lower in ADHD. It et al., 1988; Johnson and Bickel, 2002) (Fig. 3). In other words, a re-
inforcer must rapidly follow stimuli, or actions, for strengthening of
R
should also be inversely correlated with variability in RTs since both

are affected by the NA-profile in ADHD. behaviour to occur in ADHD.
4.2.6. Predictions of developmental effects from altered

CO
reinforcement learning
The BGM is concerned with direct effects of DA and NA modula-
tion and does not discuss or clearly imply effects secondary to altered
RL (Fig. 2).
4.3. The dynamic developmental theory

UN
4.3.1. Biological presumptions

The Dynamic developmental theory (DDT) (Sagvolden et al.,
2005) posits reduced tonic DA in ADHD, possibly due to reduced pre-
frontal glutamatergic drive (Grace, 2001; Solanto et al., 2001b).
Reduced tonic DA also leads to a reduced positive phasic DA re- Fig. 3. Effect of reinforcer as a function of delay between response and reinforcement
sponse in ADHD. Support for this hypothesis is derived from an ex- delivery. With a normal delay-of-reinforcement-gradient, all responses RD, RA and RC
periment on brain slice preparations from the spontaneous hyperten- will be reinforced. In ADHD, only responses RA and RC will be within reach of the re-
inforcer. Figure from Sagvolden et al. (2005), reprinted with permission. Copyright ©
sive rat (SHR) model of ADHD (Russell et al., 1995).
[2005], Cambridge University Press.
Sagvolden et al., 2005). Although the DDT does not suggest that
the SHR is a fully valid or the only useful animal model of ADHD,
it should be noted that a comprehensive review (van der Kooij and
Glennon, 2007) suggests two other animal models best overlap with
findings in human ADHD. Interestingly, these models imply opposite
alterations of the DA system (hypodopaminergic and hyperdopamin-
F
ergic transmission, respectively), suggesting that several neurobiolog-
ical pathways can lead to ADHD-like behaviour. This questions the
OO
validity of neurobiological alterations in the SHR for human ADHD.
As the BGM, the DDT is also inconsistent with the tonic/phasic
model of DA (Grace, 2000, 1991), with sparse direct evidence for why
tonic/phasic model of DA should not be valid for ADHD. Thus, the
caveats presented for the BGM are also valid for the DDT. There is
also a potential inconsistency in the DDTs account of positive and
negative PEs. For positive PEs, it is implied that lower absolute lev-
els of phasic DA give rise to ADHD, while the notion of a floor-effect
PR
on phasic DA-dips in ADHD implies that the change relative to tonic
Fig. 4. Dopamine response for positive (A) and negative (B) prediction errors. Typi- DA is important for negative PEs. Other authors relate the magnitude
cally developing controls (dashed line) and ADHD (whole line). Figure from Sagvolden of negative PEs mainly to the duration of the shutdown of dopamin-
et al. (2005), after Waelti et al. (2001), reprinted with permission. Copyright © [2005], ergic neuron firing − as the range for DA-firing reductions in general
Cambridge University Press and [2001], Macmillan Publishers Ltd.
is low in primates − or possibly other transmitter systems (Glimcher,
Low tonic DA also creates a deficit in extinction in the DDT 2011; Schultz, 2013). Deficient extinction implies stronger persevera-
(Sagvolden et al., 2005). Since DA concentrations cannot drop below tion errors in ADHD, which were shown in a small empirical study by
ED
zero, the theory assumes a reduced range of phasic DA decrease rela- the authors of the DDT (Sagvolden et al., 1998). A meta-analysis of
tive to baseline when tonic DA is low (Fig. 4B). According to the the- executive function deficits in ADHD (Willcutt et al., 2005) somewhat
ory this limits the extinction of non-adaptive behaviour and learning supports this finding by finding differences in perseveration-errors on
from worse-than-expected outcomes. the Wisconsin Card Sorting Test, but the effect size was among the
Symptoms of ADHD are deduced from deficiencies in classic re- lowest of all executive function measures, and only significant in 46%
inforcement mechanisms in the DDT (Catania, 1998; Johansen et of reviewed studies.
al., 2002; see Sagvolden et al., 2005). Impaired sustained attention
CT
in ADHD is conceptualized as a lack of stimulus control over time 4.3.4. Predictions for drift-diffusion model parameters
(Catania, 1998). Important cues that appear remotely prior to rewards
will be less efficiently reinforced as predictive cues, so that e.g. 4.3.4.1. Drift rate
teacher’s instructions will be inefficiently learned to inform when at- Reduced RL implies reduced stimulus control, which results in
tentive behaviour will be subsequently rewarded. Hyperactivity is at- inattention (Sagvolden et al., 2005). This should be picked up as
tributed to reduced extinction in the DDT (Sagvolden et al., 1998). reduced drift rate, and should contribute to all types of DM tasks.
RE
When reinforcement is evoked, all behaviour close in time will be Suboptimal reinforcement and extinction in ADHD also implies that
strengthened. Normally, non-rewarded responses will be extinguished outcomes of a choice will have less impact on the expected value
and behaviour that is causally linked to rewards will be selected out. of choosing that option in future decisions. If people with ADHD
Non-rewarded actions are not extinguished in ADHD, causing “su- have difficulties in learning to differentiate between the values of op-
perfluous” behaviour that is perceived as hyperactive. The DDT also tions, this should lead to a smaller difference of evidence signals in
explains impulsivity as short inter-response intervals (Sagvolden et value-based DM, i.e. lower drift rate. Following a central study for the
R
al., 2005; 1998): Responses occurring closely together will be more DDT (Sagvolden et al., 1998), the difference in drift rate from TDCs
within reach of the steep reinforcement curve, meaning short burst of should increase during the course of an experiment, as ADHD-par-
actions will be preferentially learned. The DDT also gives an account ticipants lag more and more behind TDCs in acquiring reinforcement
CO
of impulsivity in the sense of delay aversion (Solanto et al., 2001a; contingencies. Since this should be evident within the time frame of a
Sonuga-Barke, 2003). Cues signalling which temporally distant be- task, this is still considered a direct effect of ADHD on the drift rate
haviours will be rewarded are not as easily learned in ADHD, mak- parameter. Finally, frequent reinforcement should ameliorate differ-
ing it difficult to grasp the relationship between actions and their dis- ences between ADHD and TDCs (Sagvolden et al., 2005; p. 414).
tant consequences. Aversion to delay is thought to arise from feelings
of non-mastery in such situations, which can generalise to similar set- 4.3.4.2. Boundary separation
tings (Sagvolden et al., 2005). A steepened delay-of-reinforcement gradient in ADHD leads to
UN
selective reinforcement of response chains with short inter-response

4.3.3. Limitations and discussion of the theory times (Sagvolden et al., 2005, 1998). Responses that are learned in
According to a thorough review on altered reinforcement sensitiv- ADHD should manifest as quick responding, which is reflected in a
ity in ADHD (Luman et al., 2010) there is some evidence for some lower boundary separation. Fast responding leads to more errors, but
of the behavioural predictions of the DDT, especially a stronger dis- the extinction deficit of the DDT implies that a fast responses mode
counting of future rewards, and to some extent impaired RL in ADHD. will not be effectively extinguished. Again, the difference from TDCs
As for neurobiological predictions, the DDT leans heavily on work should be larger at the end of a task, as TDCs purge away responses
with the SHR-model of ADHD, as it displays impulsivity, inatten- that are inappropriate or prematurely timed, and deficient extinction
tion and unstable performance compared to control rats, as well as slows this purging in ADHD. This should also be evident within the
hyperactivity that diminishes with DA agonism (Sagvolden, 2000; time frame of a normal task, and consequently is deemed here as a di
rect component of ADHD. Also for boundary separation, frequent Adolescents with ADHD also show lower reward-anticipatory neural
reinforcement should reduce differences between ADHD and TDCs activity (Scheres et al., 2007) and there is behavioural evidence com-
(Sagvolden et al., 2005, pp. 412–414). patible with alterations of reinforcement sensitivity (Tripp and Alsop,
2001, 1999).
4.3.5. Predictions for PERL parameters The DTD theory assumes that DA response to immediate rewards
is normal in ADHD. In support, the authors cite behavioural studies
F
4.3.5.1. Learning rate suggesting equal performance in ADHD with continuous reinforce-
The DDT clearly implies a lower learning rate for both positive ment (Freibergs and Douglas, 1969; Parry and Douglas, 1983).
OO
and negative PEs in ADHD (Sagvolden et al., 2005). A lower pha-
sic DA response leads to weaker reward PEs, so that the expected re- 4.4.2. Theory description
ward approaches the actual average payoff more slowly. Following The DTD theory of Tripp and Wickens (Tripp and Wickens, 2009,
Fig. 3, a lower learning rate in ADHD could be found to some degree 2008) is also based on deficits in the neural processes of behaviour
even when reward delivery is immediate. Also, the floor for negative reinforcement. DA release in the striatum must take place within a
DA deflections, which puts a limit to the learning from strong nega- sub-second time window if an active synapse is to be strengthened
tive PEs, is interpreted to lead to a tendency to overestimate expected and associative learning is to occur (Black et al., 1985; Reynolds and
value, especially when losses are distributed as few and large instead Wickens, 2002; Tsai et al., 2009). On the behaviour level, learning
PR
of small and frequent ones (as in the Iowa gambling task), even if the is also inversely related to the delay between a cue or response and
overall probability-weighted loss is the same. reward delivery, but the delays in question are on a larger timescale
(Tripp and Alsop, 2001).
4.3.5.2. Choice sensitivity parameter This temporal discrepancy between reinforcement on the cellular
Selection of response chains with short inter-response times and and on the behavioural level is explained through reward predictive
less purging of inappropriate responses (Sagvolden et al., 2005, 1998) cues (actions can also be thought of as cues in this context), which can
should lead to faster and more impulsive/erratic choosing, with dif- “bridge” the delay between a cue and reinforcement. Such bridging
ED
ferences from TDCs increasing with time on a learning task. This cues work as “stand-in” secondary reinforcers and reduce the negative
should manifest as less reward-driven/more exploratory behaviour, effect of reward deferral on learning (Cardinal et al., 2004; e.g. Tarpy
which should result in lower choice sensitivity in ADHD. As this de- and Sawabini, 1974).
pends on the short ADHD-delay-of-reinforcement gradient, frequent Bridging is thought to be due to a transfer of the phasic DA re-
rewards might also compensate here and reduce the difference from sponse from reward to reward-predictive cues during the course of
TDCs (Sagvolden et al., 2005, pp. 412–414). learning (Schultz, 2002). In real life-situations there are usually a
multitude of stimuli that can function as bridging secondary rein-
CT
4.3.6. Predictions of developmental effects from altered forcers and thus provide cellular reinforcement during reward de-
reinforcement learning lays (Winstanley et al., 2004). The DTD theory assumes this trans-
fer process from reward to predictive cue to be absent or severely
4.3.6.1. Drift rate impaired in ADHD, leading to ineffective cellular reinforcement dur-
Slower acquisition of the chain of cognitive operations required for ing reward delays or incidents of reward omission (Fig. 5) (Tripp and
the task at hand implies slow, incomplete and lapsing information pro- Wickens, 2009, 2008). Importantly, the DA response to the actual re-
RE
cessing in ADHD. This should contribute to an overall lower drift rate ward delivery itself is thought to remain normal.
on the short, within-task time scale. But the developmental perspective In contrast to the DDT, the DTD theory predicts more rapid ex-
of Sagvolden et al. (2005) also imply a longer-term lag in the learning tinction of behaviour (Tripp and Wickens, 2009, 2008). Normally,
of more generally applicable cognitive operations and behaviour. This cues (or behaviour) can elicit DA that counteracts the DA reductions
should also reduce drift rate on a longer time scale. that follow from worse-than-expected outcomes (Pan et al., 2008) and
works to slow down the extinction process. In ADHD there is assumed
R
4.3.6.2. Boundary separation to be no such “protective” DA-release, and thus extinction should be
A tendency to quick responding and short inter-response times faster. Learning from conditioned stimuli should also be reduced in
should also pervade a longer time span than a single learning ex- ADHD, while learning from immediate rewards should be normal.
periment. Sagvolden et al. state that “[…] impulsiveness will be re-
CO
Building on a study by Cardinal et al. (2000), the authors also propose

duced as they [children] grow older, but this process is stunted in chil- that stimulants work mainly through increasing DA-response to pre-
dren with ADHD” (Sagvolden et al., 2005). Thus, the low bound- dictive cues.
ary separation in ADHD also has a developmental component in the Impaired sustained attention in ADHD is explained by the DTD
DDT framework. Generalised delay aversion (Sagvolden et al., 2005; theory, as in the DDT, as a lack of behavioural control by reward pre-
Sonuga-Barke, 2003) may contribute to more narrow boundaries. dictive cues (Tripp and Wickens, 2009, 2008). When DA release is
reduced during reward delays, reinforcement of reward-remote behav-
UN
4.4. The dopamine transfer deficit theory iour and bridging cues will be insufficient, and attentive behaviour
− if established − will be brittle and easily disrupted. Since DA re-
4.4.1. Biological presumptions lease to immediately rewarding stimuli is assumed to be normal, peo-
The Dopamine Transfer Deficit (DTD) theory (Tripp and Wickens, ple with ADHD are also thought to be vulnerable to immediately re-
2009, 2008) assumes a deficient transfer of DA response from un- warding off-task behaviours during delays. This results in activity that
conditioned rewards to conditioned stimuli/responses in ADHD. This is often labelled as hyperactive or impulsive, and also cause increased
view is mostly supported indirectly through research showing that sensitivity to local, short-term instances of reinforcement compared to
such a transfer normally takes place (Cador et al., 1989; Cardinal more global, long-term reinforcement contingencies (Tripp and Alsop,
et al., 2001, 2004; Schultz, 2002; Winstanley et al., 2004). 1999).
F
OO
Fig. 5. Transfer of dopamine signal from reward delivery to predictive cue. In typically developing controls (A) the dopamine response comes to be associated mainly with the cue
PR
after learning, but in ADHD (B) this transfer of the dopamine response to the cue fails to fully take place. Figure from Tripp and Wickens (2009), reprinted with permission. Copy-
right © [2009], Elsevier.
4.4.3. Limitations and discussion of the theory distractors and off-task-behaviour. The resulting inattention should be
Evidence for the basic neurobiological assumptions of the DTD picked up by DDM modelling as a lower (and more variable) drift rate.
theory was originally derived from normal physiology and behav- A study by one of the DTD theory’s authors (Tripp and Alsop, 1999)
ioural studies (Tripp and Wickens, 2009, 2008), thus giving mostly show that the stimulant methylphenidate (MPH) acutely reduces, but
indirect support to its neurobiological hypotheses on ADHD. Behav- does not normalise, ADHD-related reinforcement deficiencies, sup-
ED
ioural studies show an increased effect of reward delay (Tripp and porting both a direct and possibly a developmental effect of the condi-
Alsop, 2001) and of small, local reinforcement alterations (Tripp and tion. This indicates that reduced drift rate should at least have a direct
Alsop, 1999) in children with ADHD. But there is also a study (Luman component operating on the time scale of a typical behavioural exper-
et al., 2015) showing no difference in learning from continuous and iment.
probabilistic reward, which speaks against the theory. Only one study
has specifically investigated extinction in ADHD, and found slowed 4.4.4.2. Boundary separation
extinction in ADHD, contrary to the DTD theory’s prediction (Itami An abnormal sensitivity to delay of reinforcement in the DTD the-
CT
and Uno, 2002). In addition, the study by Sagvolden et al. (1998) cited ory implies that in cued instrumental learning tasks only very quick
in section 4.3.3 indicated perseveration in ADHD, supporting the no- timing of responses will allow for effective reinforcement of a cue
tion of slowed extinction. The review by Luman et al. (2010) con- in ADHD. This reinforcement-dependent selection of quick response
cludes there is some evidence supporting the RL impairments pre- timing should amount to a lowered boundary separation. However,
dicted by the DTD theory. quick responding also means sacrificing accuracy − and thus reward,
Transfer of the DA-signal to earlier predictive cues is supported thus possibly counteracting its reinforcement, especially when ex-
RE
by several studies on normal animal neurophysiology (Roesch et al., tinction is fast. Which effect predominates will likely depend on the
2007; Schultz, 2007c; Schultz et al., 1992) and seems dependent on cost of errors in an experiment; e.g. strict punishment of incorrect
specific brain areas known to be under dopaminergic influence (Cador responses could counteract a lower boundary separation through in-
et al., 1989; Cardinal et al., 2004, 2001; Winstanley et al., 2004). The creased extinction.
DTD theory (Tripp and Wickens, 2009, 2008) assumes no alteration
of tonic DA in ADHD, and does not seem be in direct conflict with 4.4.5. Predictions for PERL parameters
R
Grace’s tonic/phasic model (Grace, 2000, 1991). There is also some

empirical evidence supporting the hypothesis of a deficient DA trans- 4.4.5.1. Learning rate
fer in ADHD. For instance, lower (possibly DA-related) activity has In PERL modelling terminology, the DTD theory states that peo-
CO
been found in the ventral striatum during reward delays (Scheres et ple with ADHD have a lower learning rate for positive PEs when re-
al., 2007) and EEG-studies show attenuated activations to anticipated ward is delayed, but a normal learning rate with immediate rewards.
reward in ADHD (see Luman et al., 2010). Furukawa et al. (2014) For negative PEs, the learning rate should be higher in ADHD than
more recently demonstrated that the right ventral and left dorsal stria- in TDCs for omissions of delayed rewards, as there is insufficient
tum were more active during reward anticipation in healthy controls “protective” DA release in the delay between stimulus/response and
compared to patients with ADHD, while the patients had higher stri- predicted reward delivery. For omissions of immediate rewards there
atal activation during reward delivery than controls. This lends addi- should be no difference of bridging DA between ADHD and TDCs,
UN
tional support to the neurobiological assumptions of the DTD theory. and learning rates should be equal.
Showing both a normal learning rate when rewards are immedi-
ate and a higher-than-normal impact of immediate reward history, as
4.4.4. Predictions for drift-diffusion model parameters described in the theory, may appear contradictory. However, a sim-
ple PERL model could capture this seeming contradiction if the learn-
4.4.4.1. Drift rate ing rate parameter is allowed to decay during the course of learning
A central assumption of the DTD theory is that reward-predict- (Krugel et al., 2009). If this decay fails to happen properly in ADHD,
ing cues exert less control over cognition and behaviour, meaning that it would result in a learning rate that is comparable to TDCs in the
people with ADHD are more susceptible to immediately rewarding earlier trials (at least when rewarding is immediate), but a relatively
higher learning rate in later trials, which results in a stronger ten
dency to win-stay/loose-shift-type choosing. Basic PERL models do DA levels relative to tonic in ADHD through increases DA binding
not assume any memory or weighting of recent versus remote reward to inhibitory autoreceptors (Seeman and Madras, 2002), most impor-
history, and might not be applicable to the more complex learning tantly in the PFC (Berridge et al., 2006).
processes that are implied by the authors in comparing recent and re- Building on experiments of noise effects on e.g. learning, mem-
mote reward histories. ory recall and working memory in human ADHD and rat models, the
model assumes that moderate, continuous environmental noise (e.g.
F
4.4.5.2. Choice sensitivity parameter auditory white noise), via the perceptual system, increases neural sys-
Reinforcement-dependent selection of quick response timing tem SNR through the phenomenon of stochastic resonance (Moss et
OO
means an emphasis on speed over accuracy. This will also lead to al., 2004; Söderlund et al., 2007, 2015).
more erratic choosing and should manifest as a lower choice sensitiv-
ity parameter when modelling ADHD-participants. As for the bound- 4.5.2. Theory description
ary separation, this should be counteracted by a high error cost. The Moderate brain arousal (MBA) model (Sikstrom and
Söderlund, 2007) is a computational model building on Grace’s find-
4.4.6. Predictions of developmental effects from altered ings that tonic DA modulate phasic release through presynaptic in-
reinforcement learning hibitory autoreceptors (Grace, 1995, 1991). In ADHD, tonic DA lev-
els are assumed to be low (Volkow et al., 2009), resulting in reduced
PR
4.4.6.1. Drift rate total DA levels in absolute terms. However, low tonic DA leads to
The low ADHD-drift rate predicted by the DTD theory follows higher phasic release and also compensatory down-regulation of in-
from an emphasis on local over global reinforcement history. The term hibitory autoreceptors and/or up-regulation of post-synaptic receptor
“global” (Tripp and Wickens, 2008; p. 696) is interpreted in this re- levels and/or activity (Grace, 1995, 1991; Seeman and Madras, 2002).
view to include also more general internal models of reinforcement The MBA posits that it is the relative strength of the phasic DA release
contingencies acquired over a longer time course than the typical psy- compared to tonic DA levels that determines signal strength, which is
chological experiment. Thus, the DTD theory is interpreted here to assumed to be higher in ADHD after pre and postsynaptic modulation
ED
imply a developmental aspect of ADHD. In line with this, MPH ad- (Fig. 6).
ministration is predicted not to exert its full effect in the acute phase Compensatory changes on the receptor level are assumed to be in-
(Tripp and Alsop, 1999). However, it should be noted that MPH does sufficient to restore tonic and relative phasic DA levels to normal in
not seem to completely normalize function in ADHD, even after pro- stimulus poor contexts, resulting in a hypodopaminergic state and un-
longed administration (Molina et al., 2009). der-arousal (Sikstrom and Söderlund, 2007). However, cellular com-
pensations work to amplify the effect of relative phasic DA, resulting
4.4.6.2. Boundary separation in a steeper relationship between stimulus intensity and postsynaptic
CT
The mechanism described for direct effects on boundary separation DA effects in the MBA model, and thus an over-reactivity to environ-
should also produce habitual fast responding on a longer time scale, mental stimuli (Fig. 6).
adding a developmental component also to the lowered boundary sep- According to the MBA model, optimal cognitive functioning de-
aration. pends on a well-tuned state of moderate brain arousal (Sikstrom and
Söderlund, 2007). The term “arousal” is not explicitly defined in neu-
4.5. The moderate brain arousal model robiological terms by the authors (Sikstrom and Söderlund, 2007), but
RE
is generally used to designate an overall psychological and physio-

4.5.1. Biological presumptions logical activation level, and is related to phenomena like sleep, alert-
The MBA model assumes a low tonic DA in ADHD that results ness, motivation and stress (Aston-Jones and Cohen, 2005). Arousal
in higher phasic DA (Grace, 1995, 1991; Seeman and Madras, 2002). level is determined by an interaction between external factors and in-
Phasic DA relative to tonic levels determines postsynaptic effect, so ternal, nervous system-related factors such as DA and NA. Impor-
that relative phasic DA is elevated in ADHD, likely augmented by tant external factors are e.g. inter-stimulus intervals (ISIs) (Börger and
R
changes on the receptor level (Grace, 1995). Stimulants reduce phasic van der Meere, 2000; Wiersema et al., 2006), cognitive work-load
CO
UN
Fig. 6. Absolute vs. relative DA. Panel A shows absolute dopamine (DA) and panel B shows relative DA levels, in both as a function of stimulus frequency (f). The hypothetical state
of an f of zero corresponds to the tonic DA level, and is lower in both absolute and relative terms in ADHD. However, the relative phasic DA release, represented by the slope of the
curve in panel B, is larger in ADHD, leading to higher relative total DA when f is high. Figure from Sikström and Söderlund (2007), reprinted with permission. Copyright © [2007]
by the American Psychological Association.
(e.g. Houghton et al., 2004) or stimulus saliency (Corbetta and et al., in revision), possibly due to reduced phasic DA. Increased pha-
Shulman, 2002). Brain arousal is related to DA and NA and has an sic DA and postsynaptic DA effect should also be associated with re-
inverted U-shaped relationship with performance; both too little and ductions in presynaptic D2-receptors, as well as increased postsynap-
too much arousal reduce performance (Aston-Jones and Cohen, 2005; tic D1/D2-receptor levels or activity, to produce the effect posited by
Cools and D'Esposito, 2011; Goldman-Rakic et al., 2000; Robbins and the MBA model. Post-synaptic effects of G-protein coupled receptors
Arnsten, 2009). In the MBA model, DA increases arousal and thus like the DA receptors can in general desensitize in response to in-
F
the SNR in neurons by differentiating the neural output response to creased activation (Kandel et al., 2013; Leaney et al., 2004), support-
low-intensity background firing/neural noise vs. response to high-in- ing the general notion of alterations on the receptor level to altered
OO
tensity input (Sikstrom and Söderlund, 2007). Low tonic DA in DA. Still, there is sparse evidence for the specific changes posited by
ADHD means that people with ADHD are more dependent on exter- the MBA model. Most PET studies in ADHD indicate reduced D2-re-
nally evoked phasic DA to raise the SNR of neurons to optimal levels. ceptor levels (del Campo et al., 2011), but current methods cannot sep-
Moderate levels of featureless auditory noise are found to improve arate pre and postsynaptic components of this alteration (De Mei et al.,
cognition in ADHD (Söderlund et al., 2007) and learning in rat mod- 2009; Pivonello et al., 2007). D1-receptors are not extensively studied
els of ADHD (Söderlund et al., 2015). The MBA model explains this in ADHD (del Campo et al., 2011; Zimmer, 2009), but are found to be
through stochastic resonance (see e.g. Moss et al., 2004); a statisti- hypersensitive in Parkinson’s disease, where DA is also low (Gerfen
et al., 2002).
PR
cal phenomenon found in systems in which binary signals are detected
by comparing an activation level with a threshold value. When adding The findings that background noise can improve performance in
moderate levels of stochastic noise to such a system, slightly-too-weak ADHD has been replicated in several studies (Sikstrom and
activations will more often be subject to random fluctuations that Söderlund, 2007; Söderlund et al., 2007, 2015), and the effect is also
bring them above threshold. In other words, the system raises its SNR found in inattentive children without (confirmed) ADHD (Helps et
by improving sensitivity to signals. Excessive noise will of course be al., 2014; Söderlund et al., 2010). There are also studies showing al-
detrimental to signalling. The model suggests environmental noise is tered EEG-patterns in response to continuous environmental noise in
transmitted to the brain through the perceptual system (Söderlund et ADHD (Baijot et al., 2016) and increased activity in dopaminergic
ED
al., 2007). According to the MBA model, low-DA systems like in brain areas in healthy volunteers (Rausch et al., 2014). However, the
ADHD need higher noise levels for stochastic resonance to occur. notion that perceptual noise modulates DA release per se is not sup-
Distractibility and impulsive actions in ADHD is explained by ported by measurements in animals (Pålsson et al., 2011). A specu-
the MBA as an oversensitive DA response to environmental stimuli lative explanation compatible with fMRI findings could be that noise
(Sikstrom and Söderlund, 2007). Impulsive choosing, preference for works through affecting GABAergic or glutamatergic cells that also
immediacy, delay aversion and hyperactivity can also be explained as inhabit the midbrain dopaminergic nuclei (Nair-Roberts et al., 2008).
attempts to compensate for low-arousing environments by increasing One meta analysis (Szalma and Hancock, 2011) also reports mixed
CT
reward rate or decreasing inter-stimulus intervals, thus increasing DA evidence for the hypothesis that auditory noise affects arousal. Thus
release and arousal. Finally, the MBA model suggests that seemingly it may also be that environmental noise works through other mecha-
insignificant differences in task characteristics could explain contra- nisms, e.g. it may mask external distracting stimuli, which people with
dictory findings in the ADHD-literature (Sikstrom and Söderlund, ADHD are may be particularly sensitive to (Barkley, 1997; Sikstrom
2007). Arousing aspects of tasks (like inter-stimulus intervals, contin- and Söderlund, 2007).
uous environmental noise and distractors) can influence results, and
RE
should be experimentally controlled for.

4.5.4. Predictions for drift-diffusion model parameters
4.5.3. Limitations and discussion of the theory
4.5.4.1. Drift rate
The MBA model has its strengths in that it explains apparently
According to the MBA model, low-arousing tasks cause reduced
contradictory results in ADHD research. Research that indicates both
neural SNR and results in inattention to task-relevant information in
increased and decreased DA-activity in ADHD (del Campo et al.,
R
ADHD (Sikstrom and Söderlund, 2007). This should manifest as a

2011; Volkow et al., 2009; Zimmer, 2009) could (in addition to e.g.
low drift rate, especially when this happens in the cortex. The MBA
differences in age of research subjects, medication history, smoking
history, comorbidities or type of radiotracers used) be due to differ- model also predicts that moderately arousing task properties and/or
contextual factors like auditory white noise can normalize drift rate in
CO
ent arousing properties of task conditions (Sikstrom and Söderlund,

ADHD. Still, too highly arousing tasks or environmental distractors
2007). Although a wide range of findings appear consistent with the
should evoke large phasic DA release, leading to distractibility and too
MBA model, it is falsifiable, as was also shown by model fitting
much attention to task-irrelevant information, which again results in a
experiments (Sikstrom and Söderlund, 2007). However, conclusive
low drift rate.
experiments will require comprehensive comparisons of ADHD and
control groups across varying task conditions (see e.g. Huang-Pollock
et al., 2016). People with ADHD should perform worse than TDCs 4.5.4.2. Boundary separation
UN
when task characteristics are in the outer ends of the spectrum, e.g. As for the drift rate, boundary separation could also show an in-
with very long or very short inter-stimulus intervals, or with low verted U-shaped relationship with arousing task- and contextual fac-
or high cognitive work-load or stimulus saliency (Sikstrom and tors in ADHD. Low-arousing tasks with e.g. long inter-stimulus in-
Söderlund, 2007). tervals, low rewards or low-saliency stimuli should induce compen-
The hypothesised decreased tonic and increased phasic DA in satory fast choosing to increase DA and arousal levels. This translates
ADHD is also supported by the recent PET study of Badgaiyan et al. to a lower boundary separation. Optimal/moderate arousal could bring
(2015). A currently unpublished reanalysis of data from Frank et al. boundary separation (close) to TDC levels, while highly arousing task
(2007a) also finds that MPH actually reduces learning rates (Pedersen aspects should invoke large phasic DA-release, fast and error-prone
impulsive choosing (Sikstrom and Söderlund, 2007; p. 1050), that is
again reflected in a low boundary separation.
4.5.5. Predictions for PERL parameters sults from five other studies, as well as EZ-diffusion2 meta-analyses
on children (Huang-Pollock et al., 2012) and adults (Mowinckel et al.,
4.5.5.1. Learning rate 2015). The selection of studies indicates that diffusion modelling is a
Given that external factors affect DA release, the hypersensitive fairly recent and growing approach in ADHD. Main findings of the
DA system in ADHD implies that learning rates for rewards can both studies are summarized in Table 2. Although studies interpret DDM
be increased, normal and reduced. Moderately arousing task contin- results from different theoretical viewpoints, the following discussion
F
gencies, such as moderate ISIs, evoke phasic DA release that can will focus on relating findings to the ADHD theories reviewed in this
raises tonic levels. This can again increase the stimulation of in- paper.
OO
hibitory autoreceptors, giving normalised phasic DA and thus normal
learning rates in ADHD. With highly arousing contingencies (equiva- 5.1.1. Drift rate in theories and empirical studies
lent to a high stimulus frequency in Fig. 6B), tonic DA could rise to In line with all reviewed theories, a clear majority of empirical
a level where phasic DA is actually reduced, thus giving lower posi- studies find a reduced drift rate in ADHD, with moderate to large ef-
tive prediction errors and learning rates. Conversely, if arousal is low, fect sizes found in all three meta-analyses (Huang-Pollock et al., 2012;
phasic DA is elevated, giving high learning rates. It should be noted Karalunas et al., 2014; Mowinckel et al., 2015).
that a very high learning rate can also be suboptimal, as this leads to The literature discusses whether a reduced drift rate is due to diffi-
highly variant expected values and a short-sighted win-stay/loose-shift culties in basic information processing abilities, or in higher-level ex-
PR
choice strategy. ecutive functions (Salum et al., 2014a). Karalunas et al. (2013) found
A brisk DA increase to rewards can also give a larger contrast be- that a lower drift rate partially mediated deficits in executive functions
tween phasic DA levels and negative DA dips, meaning that reward like working memory and inhibition in ADHD. Even more clearly,
omissions will have a relatively higher impact on people with ADHD Salum et al. (2014a) found basic processing deficiencies (as opera-
(Sikstrom and Söderlund, 2007; p. 1066). This also implies that re- tionalized by drift rate on a low cognitive load DM task) to fully ac-
ward omissions should have normal impact in moderately arousing count for alterations in inhibitory-based executive functioning. Ba-
settings, while in learning rate for negative PEs should be high in stim- sic processing alterations were also found to be specific for ADHD,
ED
ulus-poor environments, and low in highly arousing environments. and separated this group from other clinical conditions (Salum et al.,
2014a). An alteration of basic information processing is in line with
neurobiological theories reviewed herein, as none solely implies a de-
4.5.5.2. Choice sensitivity parameter ficiency in higher-order cognitive functions in ADHD.
Low arousal and total DA lead to reduced brain SNR and thus less Although a low drift rate is most commonly viewed as a marker
contrast between representations of options in the cortex (Sikstrom of task difficulty and basic information processing ability, Merkt et al.
and Söderlund, 2007). This implies that choices will be erratic and (2013) also use drift rate alterations as an index of different attention
CT
the choice sensitivity parameter low in ADHD. High relative phasic strategies. In female students with ADHD, a reduced congruency ef-
DA responses could also make for impulsive choosing/a low boundary fect on drift rate in a flanker task was interpreted as a sign of more
separation, and thus contribute to a low choice sensitivity parameter. narrowly focused attention. This strategy reduced interference from
Again, this should especially be the case when the task and environ- incongruent flankers, but also precluded ADHD-participants from ex-
mental conditions are not very stimulating and tonic DA is low. Un- ploiting the information from congruent flankers. This is inconsistent
der optimally arousing conditions, the choice sensitivity parameter is with the MBA model, which assumes distractors such as incongruent
RE
interpreted to normalize in the MBA framework, while highly arous- flankers withdraw attention through evoking phasic DA (Sikstrom and
ing conditions could increase relative DA to a level that reduces brain Söderlund, 2007). However, Merkt et al.’s (2013) sample of college
SNR (Cools and D'Esposito, 2011; Goldman-Rakic et al., 2000), and students is highly selected, and likely not typical of ADHD. The cau-
thus choice sensitivity. tious response pattern gave students with ADHD a lower error rate
than typical students, but was also associated with obsessive-compul-
sive traits, suggesting they had developed a strict self-control to com-
R
4.5.6. Predictions of developmental effects from altered pensate for their ADHD (Merkt et al., 2013).
reinforcement learning The DDT predicts increasing differences in drift rate through the
The MBA model is mainly concerned with more or less immediate course of tasks, as ADHD-participants lag more and more behind
DA-effects on brain SNR; so developmental effects from learning that TDCs in learning task contingencies and strategies. However, when
CO
should affect DDM parameters are not clearly implied by the theory analyzing the Supplementary data from Huang-Pollock et al. (2012),
(Sikstrom and Söderlund, 2007). the correlation of task length and the between-group difference in
drift rate was not significant (r = 0.002, p = 0.99; based on 10 stud-
ies for which the required data were available). Long ISIs were asso-
5. Theory predictions and empirical modelling studies ciated with faster ADHD drift rates (r = 0.89, p < 0.001), contrary to
the MBA model’s assumptions of long ISIs being associated with sub-
5.1. Drift-diffusion modelling optimal arousal. This finding also appears inconsistent with the DDT
UN
and DTD theories, if one assumes that long ISIs are associated with
A search in the PubMed and ISI Web of Science databases May reduced stimulus control, thus leading to low drift rates.
2016 using the search terms <(ADHD OR attention deficit* OR at- A recent study (Huang-Pollock et al., 2016) which is close in
tention-deficit*) AND (drift diffusion OR drift-diffusion OR diffu- spirit to this review used the DDM to compare theories of ADHD.
sion model)> returned 27 and 63 hits, respectively. Abstracts of the The tested theories were the Cognitive energetic theory (Sergeant,
hits were screened and 13 relevant articles were identified. This in- 2000; 2005) and the Neuroenergetics theory (Killeen et al., 2013),
cluded one meta-analysis (Karalunas et al., 2014) that compiled re
2
The EZ-diffusion model (Wagenmakers et al., 2007) gives a rough estimate of
drift rate and boundary separation using aggregated data.
Table 2 Null findings could − at least partially − be due to small effects of

Overview of findings from empirical Drift-Diffusion Model (DDM) literature. Drift rate ADHD on boundary separation, at least for some tasks. Salum et al.
and boundary separation are relative to typically developing controls. NA = not applic-
able due to task difficulty levels being individually adjusted. 1 = no significant overall (2014a) found a higher boundary separation in ADHD on a two-choice
group difference, but congruency effect of flanker task (i.e. increased drift) rate less in RT task with the exact same task properties as in Metin et al. (2013),
ADHD, and frequent conflicting trials did not increase drift rate in ADHD. 2 = trend to- which found no difference. However, Salum et al. (2014a) showed
wards higher. 3 = lower drift rate on “go”-trials with slow event-rats and on “no-go”-tri- a small effect size (Cohen’s d = 0.272, 95% CI: 0.051–0.493) in a
F
als with fast event-rates, otherwise no difference. 2C-RT = Two-Choice Reaction Time
Task. CCT = Conflict Control Task.
large sample (nTDC = 378, nADHD = 100), indicating that boundary al-
terations in ADHD might be so small that other studies are underpow-
OO
Drift Boundary ered for detecting them. For instande, a subset of the same sample
Article rate separation Comments showed no significant difference of boundary separation (Salum et al.,
Mulder et al. (2010) NA ↓/= Reviewed in Karalunas et al. 2014b). In addition, the confidence interval of the effect reported by
(accuracy (2014) Metin et al. (2013) (d = 0.135, 95% CI: −0.239–0.508) largely over-
trials/speed laps with the confidence interval of the effect reported by Salum et
trials) al. (2014a), suggesting a possible small difference in boundary separa-
Huang-Pollock et al. ↓ = EZ-DDM meta-analysis
(2012)
tion for this task that was not detected due to lower power (nTDC = 48,
nADHD = 65). Still, a (close to) null finding for boundary separation al-
PR
Karalunas et al. ↓ = Reviewed in Karalunas et al.
(2012) (2014) terations can be consistent with the hypothesis of higher impulsivity
Karalunas and ↓ = Reviewed in Karalunas et al. in ADHD, because a boundary separation as in TDCs can be too low
Huang-Pollock (2014)
to keep accuracy at optimal levels in ADHD patients with lower drift
(2013)
Killeen et al. (2013) ↓ ↓ Applying a model similar to the rates (see Mulder et al., 2010).
DDM Three studies show low boundary separation in ADHD (Killeen
Merkt et al. (2013) =1 ↑ Adults et al., 2013; Mulder et al., 2010; Weigard and Huang-Pollock, 2014),
Metin et al. (2013) ↓ = Reviewed in Karalunas et al. in line with all reviewed theories. The earliest study applying DDM
ED
(2C-RT and (2014)
to ADHD (Mulder et al., 2010) suggests difficulties with optimal
CCT)
Karalunas et al. ↓ = Meta analysis speed-accuracy trade-off adjustments in ADHD. Boundary separa-
(2014) tion overall was higher in trials emphasising accuracy compared to
Mowinckel et al. ↓ =2 Adults, EZ-DDM meta analysis, speed-trials, as expected, but increased less, and thus was lower for
(2014) the ADHD group compared to TDCs in accuracy trials. The Neuroen-
Weigard and Huang- ↓ ↓
Pollock (2014)
ergetics theory of Killeen et al. (2013) posits an inadequacy of neural
Salum et al. (2014a) Reviewed in Karalunas et al. energy supply in ADHD, and constructs a mathematical model sim-
CT
↓ ↑/=
(2C-RT/CCT) (2014), drug-naïve participants ilar, but no strictly comparable, to the DDM to describe this. When
Salum et al. (2014b) ↓ = Drug-naïve participants modelling data from former studies, the authors find a lower response
Huang-Pollock et al. ↓/=3 = DDM used for theory comparison criterion − the model’s analogue of boundary separation − in ADHD.
(2016)
A study on contextual cueing effects in children also found an overall
lower boundary separation in ADHD (Weigard and Huang-Pollock,
2014).
RE
which are not reviewed here due to a limited level of neurobiologi- Differences of task characteristics is another possible explanation
cal detail and a relative recency in the ADHD field, respectively. By for findings of altered boundary separation, and all reviewed theories
varying ISIs on a go/no-go-task, the study found that ADHD-partici- imply that boundary separation should vary according to task vari-
pants had a slower drift rate toward the more effortful response inhi- ables, e.g. task duration (DDT), error cost (DTD-theory) or arousing
bition option on no-go-trials when ISIs where short. This result sup- factors (MBA-model). The BGM posits a general increase of bound-
ports a depletion of resources with frequent and demanding responses, aries with increasing decision conflict, but if one assumes that the NA
R
in line with the Neuroenergetics theory, but contradicts the Cognitive deficiencies posited for ADHD disrupt subthalamic nucleus function
energetic theory, and again the MBA model, which predicts increased similarly to deep brain stimulation in Parkinson’s disease, the relation-
arousal and drift rate with faster ISIs (though it is possible that the ship between decision conflict and boundary separation might also be
CO
ISI in the study was too high for optimal arousal) (Huang-Pollock et reversed in ADHD (Cavanagh et al., 2011). Thus, one could speculate
al., 2016). The BGM’s assumption of increased NoGo-signalling in whether studies finding lower boundaries in ADHD had higher-con-
ADHD, also implies a higher drift rate for go/no-go task response in- flict decisions in the task, but such a relationship between e.g. diffi-
hibition, which is not supported by the study (Huang-Pollock et al., culty and boundary separation cannot be discerned from the analyses
2016). of these studies (Killeen et al., 2013; Mulder et al., 2010; Weigard and
Huang-Pollock, 2014). Delay aversion could also explain lower deci-
5.1.2. Boundary separation in theories and empirical studies sion boundaries, although it is not clear why the tasks applied in stud-
UN
While a clear majority of empirical studies find lower drift rates in ies showing low boundary separation should elicit more delay aver-
ADHD, the case for boundary separation is more complex. The most sion than others. Huang-Pollock et al. (2012) explored general asso-
frequent finding in the literature is no significant difference in bound- ciations between boundary separation and major task factors such as
ary separation between ADHD and TDC groups, and this was also the number of trials, stimulus display time, ISIs and length of task, but
pooled result of a meta-analysis of children (Karalunas et al., 2014). found no effects in their meta-analysis. Correlating the between-group
This is as predicted by the DTD theory when error cost is high, but difference in boundary separation and task length from available data,
as no study varied the cost of error, this prediction cannot be fully as- to assess the predictions of the DDT, also gave no significant result
sessed by the current literature. It is also possible that equal boundary (r = 0.32, p = 0.37). Still, the power to detect effects was low, so the
separation in the majority of studies is due to tasks with optimal arous- exact relationship between task factors and DDM parameters should
ing properties, as predicted by the MBA model. Still, it seems unlikely be explored further.
with such a various collection of tasks.
Neither of the theories reviewed here predict a higher boundary what from the PERL model applied herein, and since the studies were
separation for ADHD; still, this is found in two studies (Merkt et simulation studies that fitted parameters to data in a more qualitative
al., 2013; Salum et al., 2014b), and with a clear trend in a third one sense, these will not be further discussed here.
(Mowinckel et al., 2015). Changes and compensation with maturation Hauser et al. (2014) compared the fit of a PERL-like anticorrelated
is also a possible explanation for this gap between theory and observa- Rescorla-Wagner RL model with a hierarchical Gaussian filter model,
tions, as two of the studies are on adults, whereas all other studies ex- which has a learning rate that changes according to the volatility of
F
amined children (some including adolescents). With age, people with the environment as well as beliefs about stimulus values. The sim-
ADHD might learn to compensate for a low drift rate, as indicated pler Rescorla-Wagner model, which has a fixed learning rate across
OO
by findings that DSM-IV symptoms regress with age (Faraone et al., all trials, better described participants with ADHD on a probabilistic
2006). One could speculate whether this is partially trough an altered reversal learning task, indicating that they were not sensitive to subtle
speed-accuracy trade-off and thus boundary separation. In line with changes in reward contingencies. A learning rate that is less sensitive
this, hyperactivity/impulsivity symptoms, which are inversely corre- to beliefs and the volatility of the environment is reminiscent of the
lated to boundary separation (Mulder et al., 2010), seem to decline DTD theory’s predominance of local over global reinforcement con-
faster during adolescence than inattention (Biederman et al., 2000). tingencies in ADHD (Tripp and Wickens, 2009, 2008). Analyses also
Merkt et al.’s (2013) study also suggests the possibility of compen- revealed that ADHD participants could not be characterized by an al-
satory caution, as well as focusing strategies picked up by drift rate, as tered learning rate per se, but exhibited a more exploratory choice be-
PR
explained above. However, a comparison of error rates in Mowinckel haviour (Hauser et al., 2014), as modelled by a softmax choice rule
et al. (2015) suggest compensations such as higher boundary separa- like that of the PERL framework applied in this article.
tion are not enough to fully normalize CPT accuracy. The DDT is the Agay et al. (2010) applied the Expectancy Valence model
only theory reviewed here with an explicit developmental perspective, (Busemeyer and Stout, 2002), which expands on a similar framework
though the DTD theory is also interpreted to imply developmental al- as the PERL model described here,3 to ADHD-performance on the
terations. These theories imply that developmental factors will con- Iowa gambling task (Bechara et al., 1994), but found no differences
tribute to a reduced boundary separation compared to TDCs, which from TDCs. An RL study by Luman et al. (2009) did not use computa-
ED
currently has only weak support. tional models of RL, but the results indicate comparable learning rates
In sum, the literature agrees with the theory predictions of a lower between ADHD subjects and TDCs, supporting the findings of Hauser
drift rate in ADHD, with moderate to large effect sizes. Drift rate et al. (2014).
could be specific for ADHD (Salum et al., 2014a) and mediate defi- In sum, the found studies imply no alteration of learning rate in
cient executive functions (Karalunas and Huang-Pollock, 2013; Salum ADHD. Hauser et al. (2014), also provided evidence that a simpler
et al., 2014a), it increases with long ISIs (Huang-Pollock et al., 2012) learning mechanism better described RL in ADHD, and that perfor-
and is slower for response inhibition (Huang-Pollock et al., 2016). The mance differences were more related to choice sensitivity than learn-
CT
findings for boundary separation are more mixed, with null findings ing abilities. So although the literature applying PERL models is
being most frequent. This might be attributable to low power, at least sparse, empirical studies imply that the problem in ADHD seems not
for some tasks, therefore more carful experimentation is needed to bet- to be learning, but applying learned reward expectations to optimize
ter understand speed-accuracy trade-offs in ADHD. Boundary separa- reward. No theory is preferentially supported by PERL study results.
tion, reflecting adjustments to optimize DM, should in theory be more
sensitive to differences in task-related factors such as difficulty or in- 6. Summary and discussion
RE
centive structure. Thus it might be viewed more as a useful indicator

of difficulties in cognitive adjustment and effort/energy allocation in The aim of this paper has been to provide a critical review of the
ADHD (Mulder et al., 2010). It might also indicate strategies learned pathogenic DM and RL mechanisms described or implied by current
to compensate for a slow processing speed/drift rate in ADHD. Still, neurobiologically detailed theories of ADHD, as viewed through two
the hypothesis that ISIs or task length affect boundary separation, is prominent computational models of DM and RL. The neurobiological
currently not supported by the literature (Huang-Pollock et al., 2012), foundation of the theories has been briefly assessed, and their predic-
R
although studies are few. When comparing findings from empirical tions for DDM and a simple PERL model have been compared to stud-
DDM studies in ADHD to the predictions of reviewed theories, results ies applying the same (and to some extent similar) models to ADHD.
do not give clear preferential support to any particular theory. This section summarises the findings of the former sections, and also
suggests DDM/PERL experiments that can used to compare the theo-
CO
ries reviewed herein.

5.2. PERL modelling
6.1. Neurobiological basis of theories
A search in the PubMed and ISI Web of Science databases June
2016 using the search terms <(ADHD OR attention deficit* OR at- Neurobiological presumptions of the theories are briefly reviewed
tention-deficit*) AND reinforcement learning AND (learning rate OR in the sections describing each theory. All theories agree on an as-
softmax OR Q-learning OR computation* OR model*)> returned 91 sumed role of DA in ADHD pathogenesis, but differ on the finer de-
UN
(when limited to studies on humans) and 118 hits, respectively. Only tails of this. The discussion section here will focus on the core issue
one study was found that applied a learning model equivalent to the of DA alterations in ADHD, as to go fully into the details of neu-
basic PERL model to ADHD (Hauser et al., 2014). Another study us- robiological predictions and related empirical findings for ADHD is
ing a comparable model (Agay et al., 2010) was found through the beyond the scope of this review. Some predictions are reviewed by
literature reviewed in this article, as well as one displaying learn- Luman et al. (2010) for the DDT, DTD theory and BGM, and several
ing curves that give a visual impression of learning rates (Luman et
al., 2009). The search also found studies using variants of the TD
3
model framework (Cockburn and Holroyd, 2010; Silvetti et al., 2013; A search for “ADHD AND expectancy AND valence” in the above mentioned
Williams and Dayan, 2005). Since the TD framework differs some databases did not yield any additional relevant results.
other, less theoretically focused reviews of ADHD neurobiology exist ment frequency, response cost and other arousing factors than ISIs on
(G. Bush, 2009; Castellanos and Tannock, 2002; Cortese et al., 2012; DDM parameters remain to be tested.
Stanford and Tannock, 2012). No clear “winner” seems to stand out when viewing ADHD theo-
All theories base their hypothesis of a DA deficiency in ADHD to ries up against empirical DDM results. This indicates that none of the
some extent on indirect support from the known clinical efficiency of theories do fully grasp the ADHD condition, or that the experiments
stimulant medication (Faraone, 2010; Fredriksen et al., 2013; Wilens,
F
reported in the literature are not fully suitable to distinguish between
2008). Still, this is hardly conclusive evidence of an ADHD cate- theories. Of course, this could also reflect weaknesses in the interpre-
cholamine deficiency, as stimulants can have cognitive effects also in tations of theories by the authors of this article, but this might again
OO
TDCs (Agay et al., 2010), and effectiveness can depend on baseline also be an index of the inherent flexibility of the qualitatively stated
levels of DA/NA, cognitive characteristics and task type (Arnsten and hypotheses. When restating theoretical assumption in a rigid compu-
Pliszka, 2011; del Campo et al., 2013; Swanson et al., 2010). Tonic tational modelling framework, it becomes clear how apparently spe-
DA levels in ADHD have been assessed more directly with molec- cific theories still can be ambiguous. Nevertheless, the recent study
ular imaging methods, but current results still point in several direc- of Huang-Pollock et al. (Huang-Pollock et al., 2016) shows the po-
tions (del Campo et al., 2011; Luman et al., 2010; Volkow et al., 2009; tential of applying methods of analysis that does not ignore the in-
Zimmer, 2009). terdependence between RTs and accuracy − such as the DDM − for
As for phasic DA release, both the BGM and the DDT assume head-to-head testing of prominent hypothesis in the ADHD field.
PR
this is reduced in ADHD, in opposition to the tonic-phasic relationship An experiment for comparing theory predictions for DDM para-
found by Grace et al. (Grace, 2000, 1991). The arguments of the two meters could for instance use a modification of the probabilistic selec-
theories are supported by high-fidelity neurochemical studies, but in tion task of Frank et al. (2004, 2007a), as this task enables differen-
healthy animals (Ruskin et al., 2001) and with stimulant doses above tial assessment of approach and avoidance choice behaviour. Lower
what is typically used in clinical studies (Schiffer et al., 2006). The drift rate for approach behaviour and higher for avoidance would sup-
well-powered PET study of Badgaiyan et al. (2015) directly supported port the BGM, while analysing differences in drift rate and boundary
the assumption of the MBA of increased phasic DA in human ADHD separation between ADHD and TDCs during the course of the experi-
ED
during a flanker task. This is an interesting finding, and it would be ment permits assessment of the DDT’s assumptions of increasing dif-
useful to repeat such a study with a task with even stronger theoretical ferences with time on task. Having two conditions − one where incor-
connection to phasic DA release, such as an RL task. rect responses just lead to reward omissions and one where they entail
Independent of their consistency with a canonical model of the DA a small monetary loss − would allow for testing the effect of error cost
system, the nearly exclusive focus of the reviewed theories (with the on boundary separation, as implied by the DTD theory. Finally, the ex-
exception of the BGM) on DA is a potential weakness. The role of periment should have at least three levels of arousing conditions, e.g.
NA for attention is well established (Aston-Jones and Cohen, 2005), ISIs, to assess the assumptions of the MBA model.
CT
as is the interdependence of DA and NA signalling (Guiard et al.,
2008), the effect of central stimulants on the NA systems (Berridge
and Devilbiss, 2011), and the clinical effectiveness of the indirect NA 6.3. PERL-model parameters
agonist atomoxetine (Faraone, 2010). It therefore would seem that NA
should also play an important causal role in the emergence of ADHD The reviewed theories agree on reduced learning rate for positive
symptoms and associated deficits in learning and DM. The limited dis- PEs, at least under certain reinforcement/arousal conditions (see Table
RE
cussion of NA in this review is more a reflection of the reviewed the- 1). The same goes for the PERL choice sensitivity parameter. There is
ories than a conclusive indication about the role of NA for ADHD. In- more disagreement on learning from worse-than-expected outcomes.
stead, increased focus on the role for NA for ADHD might result in Also for PERL modelling parameters, all theories entail a unique con-
important new insights. stellation of parameter predictions and conditions for observing them,
thus making it possible to experimentally compare theories.
6.2. DDM parameter predictions Although RL has been broadly studied and implied for ADHD
(Luman et al., 2015), only two studies (Agay et al., 2010; Hauser et
R
The reviewed theories often assume the same direction of change al., 2014) have applied PERL models strictly comparable to the one
for DDM parameters in ADHD. All theories imply reduced drift rate, described in section 3.2. These studies find no clear evidence of alter-
at least under certain conditions, thus agreeing with the bulk of empir- ations of learning rate in ADHD, contrary to the predictions of all the-
CO
ical findings. All theories also imply that boundary separation can be ories reviewed herein. This event though the Iowa gambling task ap-
lower in ADHD, an assumption that is only supported by a few studies plied by Agay et al. (2010) is an ideal task ideal for evaluating whether
(Killeen et al., 2013; Mulder et al., 2010; Weigard and Huang-Pollock, losses have a higher impact (as implied by the BGM, as well as the
2014). The DTD theory and the MBA assume decision boundaries can DTD theory and MBA model under certain conditions), or a lower im-
be comparable to TDCs, at least under some conditions, in line with pact (as implied by the DDT) on choice behaviour on behaviour in
the majority of empirical studies, that find no effect on boundaries of ADHD.
ADHD (though no strong theoretical conclusions can not be drawn As for modulatory conditions, Hauser et al. (2014) do not provide
UN
from null findings). information on ISIs, so it remains open whether this was at an opti-
Still, when taking into account the implied conditions for observ- mally arousing level that could explain normal learning rates, as as-
ing altered modelling parameters in ADHD, each theory has its own sumed by the MBA model. If reward was immediate, this could ex-
distinct combination of predictions (see Table 1). This makes it pos- plain normal learning rates according to the DTD theory, but reward
sible to discriminate between theories using DDM modelling. Some was also probabilistic, which according to the theory should result in
of the modulatory conditions for DDM parameters (see Table 1 and higher extinction (Tripp and Wickens, 2008). The analysis of Hauser
section 5) have already been explored in the literature, e.g. the effect et al. (2014) did not separate learning rates for positive and negative
of inhibitory responding, task length and ISIs (Huang-Pollock et al.,
2012, 2016). The effect of approach/avoidance behaviour, reinforce
PEs, but faster extinction implies a higher average learning rate if the The idea of secondary developmental effects as described in this
assumptions of the DTD were right, which was not found. article is compatible with former theoretical and empirical work. In
In sum, current evidence, although sparse, indicates on the one addition to the DDT of Sagvolden et al. (2005), the idea that ADHD
hand normal updating of expectations, assuming that the reviewed symptoms can develop secondary to more basal deficiencies is also
studies had enough power to detect such effects. By comparison, there at the core of the influential Dual Pathway theory of Sonuga-Barke
is clearer evidence for more erratic/exploratory choosing (Hauser et (2003). A similar indirect mechanism has also been proposed for
F
al., 2014) in ADHD during instrumental learning. Related, a recent re- anti-depressants (Harmer, 2013). Here, the idea is that medication im-
view with a computational perspective on ADHD (Hauser et al., 2016) mediately reduces biases of attention and memory to emotionally neg-
OO
ties this to the gain of neural signalling, similarly to the MBA model, ative stimuli, but that this gives rise to clinical effects first after a pe-
and posits this as the core deficiency in ADHD. An increased ten- riod of interacting with one’s surroundings with altered cognitive in-
dency for exploration in ADHD could explain inattention, impulsiv- clinations. Medicated patients need to learn over some time that the
ity and hyperactivity as increased switching between cognitive, mo- world might not be such a depressing place after all.
tivational and motoric programs. Increased exploration also fits well Both acute and long-term, possibly developmental effects of stim-
with the earlier proposed relationship between behavioural flexibil- ulant medication have been studied empirically. Placebo-controlled
ity and an increased D2:D1 signalling ratio (Durstewitz and Seamans, studies support the notion of acute effects of stimulants on ADHD,
2008; Grace et al., 2007). Of note, an increased D2-drive is hypothe- e.g. on math performance (Grizenko et al., 2013) or on activity level,
PR
sised to be caused by both high and low DA levels (Durstewitz and RTs and commission errors (Solanto and Conners, 1982). As men-
Seamans, 2008). Also, erratic/exploratory choosing can result both tioned in section 5.1, the DDM studies of Merkt et al. (2013) and
from alterations in DDM drift rate and boundary separation. Com- Mowinckel et al. (2015) also show parameter alterations in adults that
bining the DDM with a PERL model could disentangle the contribu- could indicate developmental alterations. A review of follow-up stud-
tion of boundary separation and drift rate on the low choice sensitivity ies (Fredriksen et al., 2013) concludes that initial effects of ADHD
that results from erratic/exploratory DM in ADHD (Pedersen et al., in medication are maintained, or possibly increase, on the longer term.
revision). However, the evidence is still limited, and interpretation is made diffi-
ED
A learning experiment could be used to discriminate between the cult by heterogeneous study designs and low adherence in longitudinal
reviewed theories’ predictions for PERL parameters. To do so, it trials. Long-term effects on cognitive measures are disputed in other
should involve a reversal of learning contingencies to assess learn- studies (Hellwig-Brida et al., 2011).
ing rate for extinction, as an increased extinction rate in ADHD com- The distinction between acute and developmental effects has also
pared to TDCs would speak against the DDT. The experiment should been assessed directly in the same sample. A placebo-controlled study
also compare conditions with immediate and delayed rewards to as- on acute and chronic stimulant effects in drug-naïve boys with ADHD
sess a core assumption of the DTD theory. Finally, it should again vary (Rhodes et al., 2004) found acute improvements on some non-exec-
CT
arousal levels to assess the predictions of the MBA model, e.g. hav- utive function measures, while improvements on other tasks like the
ing one condition with self-paced ISIs to determine optimally arousing Go/No-go task developed first after chronic use. Similar effects were
ISIs for each participant, and conditions with for instance halved and found through other studies (Coghill et al., 2007; Rhodes et al., 2006),
doubled ISIs. supporting the notion of a differentiation between acute and chronic
stimulant effects. However, acute effects might also wane after some
time, possibly due to medication tolerance (Coghill et al., 2007). Also,
RE
6.4. Direct vs. developmental effects there seem to be no clear pattern in these studies of step-wise improve-
ments from baseline to acute to chronic medication, as is implied by
A final important difference between some of the reviewed theo- the DDT and the DTD theory.
ries is whether changes in DM − and thus DDM parameters − are im- To properly investigate the contribution of acute and general de-
plied to arise directly from altered neurobiology, or secondarily via al- velopmental effects on DDM parameters in ADHD, one could apply
tered reinforcement processes. Among the reviewed theories, the DDT an extended placebo controlled cross-over design, where group 1 after
R
of Sagvolden et al. (2005) and the DTD theory of Tripp and Wickens the cross-over phase continues with several weeks on stimulant drugs,
(2009, 2008) seem most clearly to imply secondary developmental ef- which are concluded by a third and final test on drugs. Group 2 should
fects to be a contributing factor, both to a reduced drift rate and to a continue with several weeks on placebo, concluded by a third and final
lowered boundary separation in ADHD. test with acute drug administration. The presence of a developmen-
CO
It is known that ADHD changes during the course of development tal effect can then be tested by comparing the performance of the two
from childhood to adulthood, both in terms of symptoms (Biederman groups at the third test. An acute effect of stimulants is estimated from
et al., 2000) and brain structure (Frodl and Skokauskas, 2012; Spencer the cross-over design, while the relative strength of the developmental
et al., 2013). If the clinical development of ADHD is in part secondary effect is obtained by comparing the acute and the developmental ef-
to DA-dependent learning deficiencies, as proposed by Sagvolden et fect.
al. (2005), this entails specific predictions for medication trials. Di-
rect effects of altered neurotransmission should be ameliorated on the 7. Conclusions and future perspectives
UN
minutes to hours scale when clinically efficient plasma levels of stim-

ulants are achieved, and should return quickly to previous levels after “Essentially, all models are wrong, but some are useful” (Box and
medication washout. Alterations of computational modelling parame- Draper, 1986).
ters should be measurable on the same time scale. Developmental or
secondary effects imply that a longer period on medication is needed The results of this article support the view that DDM and
for clinical improvement, so that new learning can take place and mal- PERL-modelling can be used to describe both ADHD symptoms and
adaptations can be unlearned. Such improvements should also be less underlying mechanistic assumptions of core neurobiological theories
sensitive to acute cessation of pharmacological treatment. of ADHD. Most theories seem to start off from the authors own
findings from behavioural studies, which are then explained based
on former neurobiological evidence. As for most psychiatric disor-
ders, neurobi
ological hypotheses are supported mainly by indirect evidence. Differ- Refined, mechanistically oriented and neurobiologically valid
ent theories of ADHD imply their own unique combination of predic- analysis tools are needed to disentangle likely multifactorial and eti-
tions for PERL and DMM parameters, and thus computational model- ologically, pathogenically and clinically complex conditions such as
ling is one possible way of discriminating between theories. The pre- ADHD.
dictions of the reviewed theories are mostly in agreement with em-
pirical work with regards to DDM drift rate and PERL choice sen- Uncited reference
F
sitivity, but most also imply alterations of boundary separation and
learning rates in ADHD, at least under some conditions, which is not Egerton et al. (2009), Frank (2006), and Sergeant (2005).
OO
clearly supported by the current literature. It is important to note that
these null finding do not necessarily indicate evidence against the the- Acknowledgements
ories, given that clinical studies have often small convenience samples
that might not be large enough to detect robust but small effects. Two The work behind this article was financed by The Research Coun-
theories imply DDM parameters alterations secondary to altered RL, cil of Norway (grant nr. 213736) and the South-Eastern Norway
which warrants exploration with computational modelling in longitu- Regional Health Authority (grant nr. 2012051). The funding bodies
dinal studies, which has not been done as of today. In sum, none of the had no involvement in the design, analyses, writing or submission of
theories assessed here are in full agreement with the results of empiri- this article. Also thank you to Prof. Göran Söderlund for valuable dis-
PR
cal modelling studies. cussions on interpretations on the MBA model, and to Dr. Inge Groote
Section 6 suggests experiments to discriminate between theories. for reading through an earlier draft of this article. Potential errors are,
For the DDM predictions, this could be done with a probabilistic se- however, solely the authors’ responsibility.
lection task (Frank et al., 2004) with a condition involving punish-
ment for incorrect responses. Analyses should separate drift rates for References
approach vs. avoidance choices and look at differences in DDM para-
meters through the course of the experiment. As for all experiments, Aarts, E., van Holstein, M., Cools, R., 2011. Striatal dopamine and the interface be-
ED
there should be at least three conditions that vary an arousing factor tween motivation and cognition. Front. Psychol. 2, http://dx.doi.org/10.3389/fpsyg.
2011.00163. (article 163).
such as ISIs. To assess theories through their PERL predictions, one
Agay, N., Yechiam, E., Carmel, Z., Levkovitz, Y., 2010. Non-specific effects of
could conduct a reversal learning experiment with varying reward de- methylphenidate (Ritalin) on cognitive ability and decision-making of ADHD and
lay. Investigating developmental effects on DDM parameters could be healthy adults. Psychopharmacology (Berl.) 210, 511–519. http://dx.doi.org/10.
done through comparing acute and chronic stimulant effects on DM in 1007/s00213-010-1853-4.
an extended placebo controlled cross-over design, as described in sec- Arcos, D., Sierra, A., Nunez, A., Flores, G., Aceves, J., Arias-Montano, J.A., 2003.
Noradrenaline i ncreases the firing rate of a subpopulation of rat subthalamic neu-
tion 6.4. rones through the activation of alpha(1)-adrenoceptors. Neuropharmacol-
CT
This review has treated the different theories mainly as competi- ogy 45, 1070–1079. http://dx.doi.org/10.1016/s0028-3908(03)00315-0.
tors. However, future refined explanations of ADHD will likely have Arnsten, A.F.T., Pliszka, S.R., 2011. Catecholamine influences on prefrontal cortical
to combine aspects of different theories. For instance, the BGM and function: relevance to treatment of attention deficit/hyperactivity disorder and re-
lated disorders. Pharmacol. Biochem. Behav. 99, 211–216. http://dx.doi.org/10.
the DDT build on the same assumptions for DA alterations, and 1016/j.pbb.2011.01.020.
mainly differ in their implications for learning rate for worse-than-ex- Arnsten, A.F.T., Wang, M., Paspalas, C.D., 2015. Dopamine's actions in primate pre-
pected outcomes. Similarly, the DTD theory and MBA model are frontal cortex: challenges for treating cognitive disorders. Pharmacol.
RE
compatible in terms of neurobiological assumptions, as both enhanced Rev. 67, 681–696. http://dx.doi.org/10.1124/pr.115.010512.
Arnsten, A.F.T., 2013. The neurobiology of thought: the groundbreaking discoveries of
relative phasic DA and a DA transfer deficit can co-exist in principle. patricia goldman-Rakic 1937–2003. Cereb. Cortex 23, http://dx.doi.org/10.1093/
Especially such neurobiological compatible theories with yet varying cercor/bht195. bht195–2281.
foci (e.g. cortical SNR vs. RL) are potential candidates for the devel- Aron, A.R., Poldrack, R.A., 2006. Cortical and subcortical contributions to Stop signal
opment of integrated theories. response inhibition: role of the subthalamic nucleus. J. Neurosci. 26, 2424–2433.
Computational modelling enables rigorous hypothesis testing and http://dx.doi.org/10.1523/JNEUROSCI. 4682-05.2006.
Aston-Jones, G., Cohen, J.D., 2005. An integrative theory of locus coeruleus-norepi-
R
can also capture effects of task contingencies that do not affect tradi- nephrine function: adaptive gain and optimal performance. Annu. Rev. Neu-
tional measures. For instance, it is possible that experimental manip- rosci. 28, 403–450. http://dx.doi.org/10.1146/annurev.neuro.28.061604.135709.
ulations affect underlying neurobiological and cognitive mechanisms Börger, N., van der Meere, J., 2000. Motor control and state regulation in children with
that can be captured by drift rate or boundary separation parameters ADHD: a cardiac response study. Biol. Psychol. 51, 247–267.
CO
without giving clear, sufficiently large or straightforward effects on Badgaiyan, R.D., Sinha, S., Sajjad, M., Wack, D.S., 2015. Attenuated tonic and en-
hanced phasic release of dopamine in attention deficit hyperactivity disorder. PLoS
traditional measures like accuracy or RTs alone (Huang-Pollock et One 10, e0137326. http://dx.doi.org/10.1371/journal.pone.0137326.
al., 2016). Optimal stimulant dosage likely differs for overt behav- Baijot, S., Slama, H., Söderlund, G., Dan, B., Deltenre, P., Colin, C., Deconinck, N.,
ioural and more functionally important cognitive improvements (Hale 2016. Neuropsychological and neurophysiological benefits from white noise in
et al., 2011; Hellwig-Brida et al., 2011), and treatment response could children with and without ADHD. Behav. Brain Funct. 12, 11. http://dx.doi.org/10.
1186/s12993-016-0095-y.
also depend on subtype (Hale et al., 2011). This highlights the impor-
Barkley, R.A., Edwards, G., Laneri, M., Fletcher, K., Metevia, L., 2001. Executive
tance of developing objective and more sensitive tools for measuring
UN
functioning, temporal discounting, and sense of time in adolescents with attention

treatment response and for more valid sub-classifications of ADHD. deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD). J.
Both PERL choice sensitivity (Hauser et al., 2014) and DDM drift Abnorm. Child Psychol. 29, 541–556. http://dx.doi.org/10.1023/A:
rate (Salum et al., 2014a) provide specific computational modelling 1012233310098.
Barkley, R.A., 1997. Behavioral inhibition, sustained attention, and executive func-
markers for ADHD rooted in established computational models of RL tions: constructing a unifying theory of ADHD. Psychol. Bull. 121, 65–94. http://
and DM. Both parameters might reflect lower brain SNR due to re- dx.doi.org/10.1037/0033-2909.121.1.65.
duced cortical and/or subcortical NA and/or DA, and their relations Basten, U., Biele, G., Heekeren, H.R., Fiebach, C.J., 2010. How the brain integrates
both to neurobiological, cognitive, symptomatic and functional mea- costs and benefits during decision making. Proc. Natl. Acad. Sci. U. S.
A. 107, 21767–21772. http://dx.doi.org/10.1073/pnas.0908104107.
sures should be further explored. Bechara, A., Damasio, A.R., Damasio, H., Anderson, S.W., 1994. Insensitivity to fu-
ture consequences following damage to human prefrontal cortex. Cogni-
tion 50, 7–15. http://dx.doi.org/10.1016/0010-0277(94)90018-3.
Berridge, C.W., Devilbiss, D.M., 2011. Psychostimulants as cognitive enhancers: the Coghill, D.R., Rhodes, S.M., Matthews, K., 2007. The neuropsychological effects of
prefrontal cortex, catecholamines, and attention-Deficit/Hyperactivity disorder. chronic methylphenidate on drug-naive boys with attention-deficit/hyperactivity
Biol. Psychiatry 69, e101–e111. http://dx.doi.org/10.1016/j.biopsych.2010.06.023. disorder. Biol. Psychiatry 62, 954–962. http://dx.doi.org/10.1016/j.biopsych.2006.
Berridge, C.W., Waterhouse, B.D., 2003. The locus coeruleus-noradrenergic system: 12.030.
modulation of behavioral state and state-dependent cognitive processes. Brain Res. Cohen, M.X., Frank, M.J., 2009. Neurocomputational models of basal ganglia function
Rev. 42, 33–84. http://dx.doi.org/10.1016/s0165-0173(03)00143-7. in learning, memory and choice. Behav. Brain Res. 199, 141–156. http://dx.doi.
Berridge, C.W., Devilbiss, D.M., Andrzejewski, M.E., Arnsten, A.F.T., Kelley, A.E., org/10.1016/j.bbr.2008.09.029.
F
Schmeichel, B., Hamilton, C., Spencer, R.C., 2006. Methylphenidate preferentially Cohen, J.D., Braver, T.S., Brown, J.W., 2002. Computational perspectives on
increases catecholamine neurotransmission within the prefrontal cortex at low dopamine function in prefrontal cortex − Commentary. Curr. Opin. Neuro-
doses that enhance cognitive function. Biol. Psychiatry 60, 1111–1120. http://dx. biol. 12, 223–229.
OO
doi.org/10.1016/j.biopsych.2006.04.022. Cools, R., D'Esposito, M., 2011. Inverted-U–Shaped dopamine actions on human
Berridge, K.C., 2012. From prediction error to incentive salience: mesolimbic compu- working memory and cognitive control. Biol. Psychiatry 69, e113–e125. http://dx.
tation of reward motivation. Eur. J. Neurosci. 35, 1124–1143. http://dx.doi.org/10. doi.org/10.1016/j.biopsych.2011.03.028.
1111/j. 1460-9568.2012.07990. x. Corbetta, M., Shulman, G.L., 2002. Control of goal-directed and stimulus-driven atten-
Biederman, J., Faraone, S.V., 2002. Current concepts on the neurobiology of Atten- tion in the brain. Nat. Rev. Neurosci. 3, 201–215. http://dx.doi.org/10.1038/
tion-Deficit/Hyperactivity Disorder. J. Atten. Disord. 6 (Suppl. 1), S7–16. nrn755.
Biederman, J., Faraone, S.V., 2005. Attention-deficit hyperactivity disor- Cortese, S., Kelly, C., Chabernaud, C., Proal, E., Di Martino, A., Milham, M.P.,
der 366, 237–248. http://dx.doi.org/10.1016/S0140-6736(05)66915-2. Castellanos, F.X., 2012. Toward systems neuroscience of ADHD: a meta-Analysis
Biederman, J., Mick, E., Faraone, S.V., 2000. Age-dependent decline of symptoms of of 55 fMRI studies. Am. J. Psychiatry 169, 1038–1055. http://dx.doi.org/10.1176/
PR
attention deficit hyperactivity disorder: impact of remission definition and symp- appi.ajp.2012.11101521.
tom type. Am. J. Psychiatry 157, 816–818. http://dx.doi.org/10.1176/appi.ajp.157. Cortese, S., 2012. The neurobiology and genetics of Attention-Deficit/Hyperactivity
5.816;pageGroup:string:Publication. Disorder (ADHD): What every clinician should know. Eur. J. Paediatr. Neu-
Biederman, J., Petty, C.R., Evans, M., Small, J., Faraone, S.V., 2010. How persistent is rol. 16, 422–433.
ADHD? A controlled 10-year follow-up study of boys with ADHD. Psychiatry Cui, G., Jun, S.B., Jin, X., Pham, M.D., Vogel, S.S., Lovinger, D.M., Costa, R.M.,
Res. 177, 299–304. http://dx.doi.org/10.1016/j.psychres.2009.12.010. 2013. Concurrent activation of striatal direct and indirect pathways during action
Bilder, R.M., Volavka, J., Lachman, H.M., Grace, A.A., 2004. The catechol-O-methyl- initiation. Nature 494, 238–242. http://dx.doi.org/10.1038/nature11846.
transferase polymorphism: relations to the tonic-phasic dopamine hypothesis and del Campo, N., Chamberlain, S.R., Sahakian, B.J., Robbins, T.W., 2011. The roles of
neuropsychiatric phenotypes. Neuropsychopharmacology 29, 1943–1961. http:// dopamine and noradrenaline in the pathophysiology and treatment of atten-
dx.doi.org/10.1038/sj.npp.1300542. tion-Deficit/Hyperactivity disorder. Biol. Psychiatry 69, E145–E157. http://dx.doi.
ED
Black, J., Belluzzi, J.D., Stein, L., 1985. Reinforcement delay of one second severely org/10.1016/j.biopsych.2011.02.036.
impairs acquisition of brain self-stimulation. Brain Res. 359, 113–119. http://dx. del Campo, N., Fryer, T.D., Hong, Y.T., Smith, R., Brichard, L., Acosta-Cabronero, J.,
doi.org/10.1016/0006-8993(85)91418-0. Chamberlain, S.R., Tait, R., Izquierdo, D., Regenthal, R., Dowson, J., Suckling, J.,
Bogacz, R., Wagenmakers, E.-J., Forstmann, B.U., Nieuwenhuis, S., 2010. The neural Baron, J.-C., Aigbirhio, F.I., Robbins, T.W., Sahakian, B.J., Müller, U., 2013. A
basis of the speed-accuracy tradeoff. Trends Neurosci. 33, 10–16. http://dx.doi.org/ positron emission tomography study of nigro-striatal dopaminergic mechanisms
10.1016/j.tins.2009.09.002. underlying attention: implications for ADHD and its treatment.
Bogacz, R., 2007. Optimal decision-making theories: linking neurobiology with behav- Brain 136, 3252–3270. http://dx.doi.org/10.1093/brain/awt263.
iour. Trends Cogn. Sci. 11, 118–125. http://dx.doi.org/10.1016/j.tics.2006.12.006. De Mei, C., Ramos, M., Iitaka, C., Borrelli, E., 2009. Getting specialized: presynaptic
CT
Box, G.E.P., Draper, N.R., 1986. Empirical Model-building and Response Surface. and postsynaptic dopamine D2 receptors. Curr. Opin. Pharmacol. 9, 53–58. http://
John Wiley & Sons, New York, NY. dx.doi.org/10.1016/j.coph.2008.12.002.
Brodal, P., 2010. The Central Nervous System, 4th ed. Oxford University Press, New Doya, K., 2008. Modulators of decision making. Nat. Neurosci. 11, 410–416. http://dx.
York, NY. doi.org/10.1038/nn2077.
Busemeyer, J.R., Stout, J.C., 2002. A contribution of cognitive decision models to clin- Dreyer, J.K., Herrik, K.F., Berg, R.W., Hounsgaard, J.D., 2010. Influence of phasic
ical assessment: decomposing performance on the Bechara gambling task. Psy- and tonic dopamine release on receptor activation. J. Neurosci. 30, 14273–14283.
chol. Assess. http://dx.doi.org/10.1523/JNEUROSCI. 1894-10.2010.
Bush, R.R., Mosteller, F., 1951. A mathematical model for simple learning. Psychol.
RE
Durstewitz, D., Seamans, J.K., 2002. The computational role of dopamine D1 recep-
Rev. 58, 313–323. http://dx.doi.org/10.1037/a0017259. tors in working memory. Neural Netw. 15, 561–572. http://dx.doi.org/10.1016/
Bush, S0893-6080(02)00049-7.
2009 G. Bush, Attention-deficit/hyperactivity disorder and attention networks, Neu- Durstewitz, D., Seamans, J.K., 2008. The dual-State theory of prefrontal cortex
ropsychopharmacology (2009) http://dx.doi.org/10.1038/npp.2009.120. dopamine function with relevance to catechol-O-Methyltransferase genotypes and
Cador, M., Robbins, T.W., Everitt, B.J., 1989. Involvement of the amygdala in stimu- schizophrenia. Biol. Psychiatry 64, 739–749. http://dx.doi.org/10.1016/j.biopsych.
lus-reward associations: interaction with the ventral striatum. Neuro- 2008.05.015.
science 30, 77–86. http://dx.doi.org/10.1176/appi.ajp.2011.11060940. Egerton, A., Mehta, M.A., Montgomery, A.J., Lappin, J.M., Howes, O.D., Reeves,
R
Cardinal, R.N., Robbins, T.W., Everitt, B.J., 2000. The effects of d-amphetamine S.J., Cunningham, V.J., Grasby, P.M., 2009. The dopaminergic basis of human be-
chlordiazepoxide, alpha-flupenthixol and behavioural manipulations on choice of haviors: a review of molecular imaging studies. Neurosci. Biobehav.
signalled and unsignalled delayed reinforcement in rats. Psychopharmacology Rev. 33, 1109–1132. http://dx.doi.org/10.1016/j.neubiorev.2009.05.005.
(Berl.) 152, 362–375. Faraone, S.V., Biederman, J., Mick, E., 2006. The age-dependent decline of attention
CO
Cardinal, R.N., Pennicott, D.R., Sugathapala, C.L., Robbins, T.W., Everitt, B.J., 2001. deficit hyperactivity disorder: a meta-analysis of follow-up studies. Psychol.
Impulsive choice induced in rats by lesions of the nucleus accumbens core. Sci- Med. 36, 159–165. http://dx.doi.org/10.1017/S003329170500471X.
ence 292, 2499–2501. http://dx.doi.org/10.1126/science.1060818. Faraone, S.V., 2010. A comparison of the efficacy of medications for adult atten-
Cardinal, R.N., Winstanley, C.A., Robbins, T.W., Everitt, B.J., 2004. Limbic corticos- tion-deficit/hyperactivity disorder using meta-analysis of effect sizes. J. Clin. Psy-
triatal systems and delayed reinforcement. Ann. N. Y. Acad. Sci. 1021, 33–50. chiatry 71, 754–763. http://dx.doi.org/10.4088/JCP.08m04902pur.
http://dx.doi.org/10.1196/annals.1308.004. Fayyad, J., De Graaf, R., Kessler, R., Alonso, J., Angermeyer, M., Demyttenaere, K.,
Castellanos, F.X., Tannock, R., 2002. Neuroscience of attention-deficit/hyperactivity De Girolamo, G., Haro, J.M., Karam, E.G., Lara, C., 2007. Cross?national preva-
disorder: the search for endophenotypes. Nat. Rev. Neurosci. 3, 617–628. lence and correlates of adult attention–deficit hyperactivity disorder. Br. J. Psychi-
Catania, A.C., Sagvolden, T., Keller, K.J., 1988. Reinforcement schedules: retroactive atry 190, 402–409.
UN
and proactive effects of reinforcers inserted into fixed-interval performances. J. Fiorillo, C.D., 2003. Discrete coding of reward probability and uncertainty by
Exp. Anal. Behav. 49, 49–73. http://dx.doi.org/10.1901/jeab.1988. 49-49. dopamine neurons. Science 299, 1898–1902. http://dx.doi.org/10.1126/science.
Catania, A.C., 1998. Learning, 4 ed. Prentice Hall. 1077349.
Cavanagh, J.F., Wiecki, T.V., Cohen, M.X., Figueroa, C.M., Samanta, J., Sherman, Forstmann, B.U., Dutilh, G., Brown, S., Neumann, J., von, Cramon, Ridderinkhof,
S.J., Frank, M.J., 2011. Subthalamic nucleus stimulation reverses mediofrontal in- D.Y., Wagenmakers, K.R., 2008. Striatum and pre-SMA facilitate decision-mak-
fluence over decision threshold. Nat. Neurosci. 14, 1462–1467. http://dx.doi.org/ ing under time pressure. Proc. Natl. Acad. Sci. U. S. A. 105, 17538–17542. http://
10.1038/nn.2925. dx.doi.org/10.1073/pnas.0805903105.
Chamberlain, S.R., Müller, U., Blackwell, A.D., Clark, L., Robbins, T.W., Sahakian, Frank, M.J., O'Reilly, R.C., 2006. A mechanistic account of striatal dopamine function
B.J., 2006. Neurochemical modulation of response inhibition and probabilistic in human cognition: psychopharmacological studies with cabergoline and haloperi-
learning in humans. Science 311, 861–863. http://dx.doi.org/10.1126/science. dol. Behav. Neurosci. 120, 497–517. http://dx.doi.org/10.1037/0735-7044.120.3.
1121218. 497.
Cockburn, J., Holroyd, C.B., 2010. Focus on the positive: computational simulations Frank, M.J., Seeberger, L.C., O'Reilly, R.C., 2004. By carrot or by stick: cognitive re-
implicate asymmetrical reward prediction error signals in childhood atten- inforcement learning in parkinsonism. Science 306, 1940–1943.
tion-deficit/hyperactivity disorder. Brain Res. 1365, 18–34. http://dx.doi.org/10.
1016/j.brainres.2010.09.065.
Frank, M.J., Santamaria, A., O'Reilly, R.C., Willcutt, E., 2007. Testing computational Guiard, B.P., El Mansari, M., Merali, Z., Blier, P., 2008. Functional interactions be-
models of dopamine and noradrenaline dysfunction in attention deficit/hyperactiv- tween dopamine, serotonin and norepinephrine neurons: an in-vivo electrophysio-
ity disorder. Neuropsychopharmacology 32, 1583–1599. http://dx.doi.org/10.1038/ logical study in rats with monoaminergic lesions. Int. J. Neuropsychopharma-
sj.npp.1301278. col. 11, 625–639. http://dx.doi.org/10.1017/S1461145707008383.
Frank, M.J., Scheres, A., Sherman, S.J., 2007. Understanding decision-making deficits Hale, J.B., Reddy, L.A., Semrud-Clikeman, M., Hain, L.A., Whitaker, J., Morley, J.,
in neurological conditions: insights from models of natural action selection. Philos. Lawrence, K., Smith, A., Jones, N., 2011. Executive impairment determines
Trans. R. Soc. Lond., B, Biol. Sci. 362, 1641–1654. http://dx.doi.org/10.1098/rstb. ADHD medication response: implications for academic achievement. J. Learn.
F
2007.2058. Disabil. 44, 196–212. http://dx.doi.org/10.1177/0022219410391191.
Frank, M.J., 2006. Hold your horses: a dynamic computational role for the subthalamic Harmer, C.J., 2013. Emotional processing and antidepressant action. In: Cowen, P.J.,
nucleus in decision making. Neural Netw. 19, 1120–1136. http://dx.doi.org/10. Sharp, T., Lau, J.Y. (Eds.), Behavioral Neurobiology of Depression and Its Treat-
OO
1016/j.neunet.2006.03.006. ment, Current Topics in Behavioral Neurosciences.. Springer, Berlin, Heidelberg,
Fredriksen, M., Halmøy, A., Faraone, S.V., Haavik, J., 2013. Long-term efficacy and pp. 209–222. http://dx.doi.org/10.1007/7854_2012_210.
safety of treatment with stimulants and atomoxetine in adult ADHD: A review of Hauser, T.U., Iannaccone, R., Ball, J., Mathys, C., Brandeis, D., Walitza, S., Brem, S.,
controlled and naturalistic studies. Eur. Neuropsychopharmacol. 23, 508–527. http: 2014. Role of the medial prefrontal cortex in impaired decision making in juvenile
//dx.doi.org/10.1016/j.euroneuro.2012.07.016. attention-deficit/hyperactivity disorder. JAMA Psychiatry 71, 1165–1173. http://
Freibergs, V., Douglas, V.I., 1969. Concept learning in hyperactive and normal chil- dx.doi.org/10.1001/jamapsychiatry.2014.1093.
dren. J. Abnorm. Psychol. 74, 388–395. Hauser, T.U., Fiore, V.G., Moutoussis, M., Dolan, R.J., 2016. Computational psychia-
Frodl, T., Skokauskas, N., 2012. Meta-analysis of structural MRI studies in children try of ADHD: neural gain impairments across marrian levels of analysis. Trends
and adults with attention deficit hyperactivity disorder indicates treatment effects. Neurosci. 39, 63–73. http://dx.doi.org/10.1016/j.tins.2015.12.009.
PR
Acta Psychiatr. Scand. 125, 114–126. http://dx.doi.org/10.1111/j. 1600-0447.2011. Hellwig-Brida, S., Daseking, M., Keller, F., Petermann, F., Goldbeck, L., 2011. Effects
01786. x. of methylphenidate on intelligence and attention components in boys with atten-
Furukawa, E., Bado, P., Tripp, G., Mattos, P., Wickens, J.R., Bramati, I.E., Alsop, B., tion-deficit/hyperactivity disorder. J. Child Adolesc. Psychopharma-
Ferreira, F.M., Lima, D., Tovar-Moll, F., Sergeant, J.A., Moll, J., 2014. Abnormal col. 21, 245–253. http://dx.doi.org/10.1089/cap.2010.0041.
striatal BOLD responses to reward anticipation and reward delivery in ADHD. Helps, S.K., Bamford, S., Sonuga-Barke, E.J.S., Söderlund, G.B.W., 2014. Different
PLoS One 9, e89129. http://dx.doi.org/10.1371/journal.pone.0089129. effects of adding white noise on cognitive performance of sub-, normal and su-
Garrison, J., Erdeniz, B., Done, J., 2013. Prediction error in reinforcement learning: a per-attentive school children. PLoS One 9, e112768. http://dx.doi.org/10.1371/
meta-analysis of neuroimaging studies. Neurosci. Biobehav. Rev. 37, 1297–1310. journal.pone.0112768.
http://dx.doi.org/10.1016/j.neubiorev.2013.03.023. Hill, J.C., Schoener, E.P., 1996. Age-dependent decline of attention deficit hyperactiv-
Gerfen, C.R., Surmeier, D.J., 2011. Modulation of striatal projection systems by ity disorder. Am. J. Psychiatry 153, 1143–1146.
ED
dopamine. Annu. Rev. Neurosci. 34, 441–466. Houghton, S., Milner, N., West, J., Douglas, G., Lawrence, V., Whiting, K., Tannock,
Gerfen, C.R., Miyachi, S., Paletzki, R., Brown, P., 2002. D1 dopamine receptor super- R., Durkin, K., 2004. Motor control and sequencing of boys with Atten-
sensitivity in the dopamine-depleted striatum results from a switch in the regulation‐Deficit/Hyperactivity Disorder (ADHD) during computer game play. British
tion of ERK1/2/MAP kinase. J. Neurosci. 22, 5042–5054. Journal of Educational Technology 35, 21–34. http://dx.doi.org/10.1111/j.
Giros, B., Jaber, M., Jones, S.R., Wightman, R.M., Caron, M.G., 1996. Hyperlocomo- 1467-8535.2004.00365. x.
tion and indifference to cocaine and amphetamine in mice lacking the dopamine Huang-Pollock, C.L., Karalunas, S.L., Tam, H., Moore, A.N., 2012. Evaluating vigi-
transporter. Nature 379, 606–612. http://dx.doi.org/10.1038/379606a0. lance deficits in ADHD: a meta-analysis of CPT performance. J. Abnorm. Psy-
Glimcher, P.W., 2011. Understanding dopamine and reinforcement learning: the chol. 121, 360–371. http://dx.doi.org/10.1037/a0027205.
CT
dopamine reward prediction error hypothesis. Proc. Natl. Acad. Sci. U. S. Huang-Pollock, C., Ratcliff, R., McKoon, G., Shapiro, Z., Weigard, A., Gal-
A. 108, 15647–15654. http://dx.doi.org/10.2307/41351278?ref=search-gateway: loway-Long, H., 2016. Using the diffusion model to explain cognitive deficits in
f333c4f68a84c2f60d2b7429cb4a49fb. attention deficit hyperactivity disorder. − PubMed − NCBI. J. Abnorm. Child Psy-
Gold, J.I., Shadlen, M.N., 2007. The neural basis of decision making. Annu. Rev. Neu- chol. http://dx.doi.org/10.1007/s10802-016-0151-y.
rosci. 30, 535–574. http://dx.doi.org/10.1146/annurev.neuro.29.051605.113038. Itami, S., Uno, H., 2002. Orbitofrontal cortex dysfunction in attention-deficit hyperac-
Goldman-Rakic, P.S., Muly III, E.C., Williams, G.V., 2000. D 1 receptors in prefrontal tivity disorder revealed by reversal and extinction tasks. Neurore-
cells and circuits. Brain Res. Rev. 31, 295–301. http://dx.doi.org/10.1016/ port 13, 2453–2457. http://dx.doi.org/10.1097/01. wnr.0000047687.08940.42.
S0165-0173(99)00045-4.
RE
Johansen, E.B., Sagvolden, T., 2004. Response disinhibition may be explained as an

Goto, Y., Otani, S., Grace, A.A., 2007. The Yin and Yang of dopamine release: a new extinction deficit in an animal model of attention-deficit/hyperactivity disorder
perspective. Neuropharmacology 53, 583–587. http://dx.doi.org/10.1016/j. (ADHD). Behav. Brain Res. 149, 183–196. http://dx.doi.org/10.1016/
neuropharm.2007.07.007. S0166-4328(03)00229-8.
Grace, A.A., Floresco, S.B., Goto, Y., Lodge, D.J., 2007. Regulation of firing of Johansen, E.B., Aase, H., Meyer, A., Sagvolden, T., 2002. Attention-deficit/hyperac-
dopaminergic neurons and control of goal-directed behaviors. Trends Neu- tivity disorder (ADHD) behaviour explained by dysfunctioning reinforcement and
rosci. 30, 220–227. http://dx.doi.org/10.1016/j.tins.2007.03.003. extinction processes. Behav. Brain Res. 130, 37–45.
Grace, A.A., 1991. Phasic versus tonic dopamine release and the modulation of Johnson, M.W., Bickel, W.K., 2002. Within-subject comparison of real and hypotheti-
R
dopamine system responsivity: a hypothesis for the etiology of schizophrenia. cal money rewards in delay discounting. J. Exp. Anal. Behav. 77, 129–146. http://
Neuroscience 41, 1–24. http://dx.doi.org/10.1016/0306-4522(91)90196-U. dx.doi.org/10.1901/jeab.2002. 77-129.
Grace, A.A., 1995. The tonic/phasic model of dopamine system regulation: its rele- Käenmäki, M., Tammimäki, A., Myöhänen, T., Pakarinen, K., Amberg, C., Karayior-
vance for understanding how stimulant abuse can alter basal ganglia function. gou, M., Gogos, J.A., Männistö, P.T., 2010. Quantitative role of COMT in
CO
Neurosci. Biobehav. Rev. 37, 111–129. http://dx.doi.org/10.1016/ dopamine clearance in the prefrontal cortex of freely moving mice. J. Neu-
0376-8716(94)01066-T. rochem. 114, 1745–1755. http://dx.doi.org/10.1111/j. 1471-4159.2010.06889. x.
Grace, A.A., 2000. The tonic/phasic model of dopamine system regulation and its im- Kandel, E.R., Schwartz, J.H., Jessel, T.M., Siegelbaum, S.A., Hudspeth, A.J., 2013.
plications for understanding alcohol and psychostimulant craving. Addic- Principles of Neural Science, fifth edition, 1709.
tion 95 (Suppl. 2), S119–28. Karalunas, S.L., Huang-Pollock, C.L., 2013. Integrating impairments in reaction time
Grace, A.A., 2001. Psychostimulant actions on dopamine and limbic system function: and executive function using a diffusion model framework. J. Abnorm. Child Psy-
relevance to the pathophysiology and treatment of ADHD. In: Solanto, M.V., Arn- chol. 41, 837–850. http://dx.doi.org/10.1007/s10802-013-9715-2.
sten, A.F.T., Castellanos, F.X. (Eds.), Stimulant Drugs and ADHD: Basic and Karalunas, S.L., Geurts, H.M., Konrad, K., Bender, S., Nigg, J.T., 2014. Annual re-
Clinical Neuroscience.. Oxford University Press, New York, N.Y., U.S, pp. search review: reaction time variability in ADHD and autism spectrum disorders:
UN
134–157. measurement and mechanisms of a proposed trans-diagnostic phenotype. J. Child

Grace, A.A., 2002. Dopamine. In: Davis, K.L., Charney, D., Coyle, J.T., Nemeroff, C. Psychol. Psyc. 55, 685–710. http://dx.doi.org/10.1111/jcpp.12217.
(Eds.), Neuropsychopharmacology. Lippincott Williams & Wilkins, Philadelphia Killeen, P.R., Russell, V.A., Sergeant, J.A., 2013. A behavioral neuroenergetics theory
PA, pp. 119–132. of ADHD. Neurosci. Biobehav. Rev. 37, 625–657. http://dx.doi.org/10.1016/j.
Graef, S., Biele, G., Krugel, L.K., Marzinzik, F., Wahl, M., Wotka, J., Klostermann, neubiorev.2013.02.011.
F., Heekeren, H.R., 2010. Differential influence of levodopa on reward-based Kim, J.N., Shadlen, M.N., 1999. Neural correlates of a decision in the dorsolateral pre-
learning in Parkinson's disease. Front. Hum. Neurosci. 4, 169. http://dx.doi.org/10. frontal cortex of the macaque. Nat. Neurosci. 2, 176–185. http://dx.doi.org/10.
3389/fnhum.2010.00169. 1038/5739.
Green, N., Biele, G.P., Heekeren, H.R., 2012. Changes in neural connectivity underlie Krause, K.-H., Dresel, S.H., Krause, J., Kung, H.F., Tatsch, K., 2000. Increased striatal
decision threshold modulation for reward maximization. J. Neu- dopamine transporter in adult patients with attention deficit hyperactivity disorder:
rosci. 32, 14942–14950. http://dx.doi.org/10.1523/JNEUROSCI. 0573-12.2012. effects of methylphenidate as measured by single photon emission computed to-
Grizenko, N., Cai, E., Jolicoeur, C., Ter-Stepanian, M., Joober, R., 2013. Effects of mography. Neurosci. Lett. 285, 107–110. http://dx.doi.org/10.1016/
methylphenidate on acute math performance in children with attention-deficit hy- S0304-3940(00)01040-5.
peractivity disorder. Can. J. Psychiatry 58, 632–639.
Kreitzer, A.C., 2009. Physiology and pharmacology of striatal neurons. Annu. Rev. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Dolan, R.J., 2004. Dissociable
Neurosci. 32, 127–147. http://dx.doi.org/10.1146/annurev.neuro.051508.135422. roles of ventral and dorsal striatum in instrumental conditioning. Sci-
Krugel, L.K., Biele, G., Mohr, P.N.C., Li, S.-C., Heekeren, H.R., 2009. Genetic varia- ence 304, 452–454. http://dx.doi.org/10.1126/science.1094285.
tion in dopaminergic neuromodulation influences the ability to rapidly and flexibly O'Reilly, R.C., Frank, M.J., 2006. Making working memory work: a computational
adapt decisions. Proc. Natl. Acad. Sci. U. S. A. 106, 17951–17956. http://dx.doi. model of learning in the prefrontal cortex and basal ganglia. Neural Com-
org/10.1073/pnas.0905191106. put. 18, 283–328. http://dx.doi.org/10.1162/089976606775093909.
Leaney, J.L., Benians, A., Brown, S., Nobles, M., Kelly, D., Tinker, A., 2004. Rapid Orieux, G., Francois, C., Feger, J., Hirsch, E.C., 2002. Consequences of dopaminergic
F
desensitization of G protein-gated inwardly rectifying K(+) currents is determined denervation on the metabolic activity of the cortical neurons projecting to the sub-
by G protein cycle. Am. J. Physiol. Cell Physiol. 287, http://dx.doi.org/10.1152/ thalamic nucleus in the rat. J. Neurosci. 22, 8762–8770.
ajpcell.00540.2003. (C182-91). Overtoom, C.C.E., Verbaten, M.N., Kemner, C., Kenemans, J.L., van Engeland, H.,
OO
Li, S.-C., Lindenberger, U., Sikstrom, S., 2001. Aging cognition: from neuromodula- Buitelaar, J.K., van der Molen, M.W., van der Gugten, J., Westenberg, H., Maes,
tion to representation. Trends Cogn. Sci. 5, 479–486. R.A.A., Koelega, H.S., 2003. Effects of methylphenidate, desipramine, and L-dopa
Llorente, A.M., Voigt, R.G., Jensen, C.L., Berretta, M.C., Kennard Fraley, J., Heird, on attention and inhibition in children with Attention Deficit Hyperactivity Disor-
W.C., 2006. Performance on a visual sustained attention and discrimination task is der. Behav. Brain Res. 145, 7–15.
associated with urinary excretion of norepineprhine metabolite in children with at- Pålsson, E., Söderlund, G., Klamer, D., Bergquist, F., 2011. Noise benefit in prepulse
tention-deficit/hyperactivity disorder (AD/HD). Clin. Neuropsychol. 20, 133–144. inhibition of the acoustic startle reflex. Psychopharmacology (Berl) 214, 675–685.
http://dx.doi.org/10.1080/13854040490888495. http://dx.doi.org/10.1007/s00213-010-2074-6.
Lo, C.-C., Wang, X.-J., 2006. Cortico-basal ganglia circuit mechanism for a decision Pan, W.-X., Schmidt, R., Wickens, J.R., Hyland, B.I., 2008. Tripartite mechanism of
threshold in reaction time tasks. Nat. Neurosci. 9, 956–963. http://dx.doi.org/10. extinction suggested by dopamine neuron activity and temporal difference model.
PR
1038/nn1722. J. Neurosci. 28, 9619–9631. http://dx.doi.org/10.1523/JNEUROSCI. 0255-08.
Luce, R.D., 1959. Individual Choice Behavior: A Theoretical Analysis. John Wiley 2008.
and Sons, New York. Parry, P.A., Douglas, V.I., 1983. Effects of reinforcement on concept identification in
Luman, M., Van Meel, C.S., Oosterlaan, J., Sergeant, J.A., Geurts, H.M., 2009. Does hyperactive children. J. Abnorm. Child Psychol. 11, 327–340.
reward frequency or magnitude drive reinforcement-learning in attention-deficit/ Pe, M.L., Vandekerckhove, J., Kuppens, P., 2013. A diffusion model account of the re-
hyperactivity disorder?. Psychiatry Res. 168, 222–229. http://dx.doi.org/10.1016/j. lationship between the emotional flanker task and rumination and depression.
psychres.2008.08.012. Emotion 13, 739–747. http://dx.doi.org/10.1037/a0031628.
Luman, M., Tripp, G., Scheres, A., 2010. Identifying the neurobiology of altered rein- Pedersen, M.L., Frank, M.J., Biele, G.P., 2016. The drift diffusion model as the choice
forcement sensitivity in ADHD: A review and research agenda. Neurosci. Biobe- rule in reinforcement learning. Psychonomic bulletin & review (in revision).
hav. Rev. 34, 744–754. http://dx.doi.org/10.1016/j.neubiorev.2009.11.021. Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J., Frith, C.D., 2006.
ED
Luman, M., Goos, V., Oosterlaan, J., 2015. Instrumental learning in ADHD in a con- Dopamine-dependent prediction errors underpin reward-seeking behaviour in hu-
text of reward: intact learning curves and performance improvement with mans. Nature 442, 1042–1045. http://dx.doi.org/10.1038/nature05051.
methylphenidate. J. Abnorm. Child Psychol. 43, 681–691. http://dx.doi.org/10. Pivonello, R., Ferone, D., Lombardi, G., Colao, A., Lamberts, S.W.J., Hofland, L.J.,
1007/s10802-014-9934-1. 2007. Novel insights in dopamine receptor physiology. Eur. J. En-
Maia, T.V., Frank, M.J., 2011. From reinforcement learning models to psychiatric and docrinol. 156 (Suppl 1), S13–S21. http://dx.doi.org/10.1530/eje.1.02353.
neurological disorders. Nat. Neurosci. 14, 154–162. http://dx.doi.org/10.1038/nn. Pliszka, S.R., 2005. The neuropsychopharmacology of attention-deficit/hyperactivity
2723. disorder. Biol. Psychiatry 57, 1385–1390. http://dx.doi.org/10.1016/j.biopsych.
Merkt, J., Singmann, H., Bodenburg, S., Goossens-Merkt, H., Kappes, A., Wendt, M., 2004.08.026.
CT
Gawrilow, C., 2013. Flanker performance in female college students with ADHD: Polanczyk, G.V., Willcutt, E.G., Salum, G.A., Kieling, C., Rohde, L.A., 2014. ADHD
a diffusion model analysis. Atten. Defic. Hyperact. Disord. 5, 321–341. http://dx. prevalence estimates across three decades: an updated systematic review and
doi.org/10.1007/s12402-013-0110-1. meta-regression analysis. Int. J. Epidemiol. 43, 434–442. http://dx.doi.org/10.1093/
Metin, B., Roeyers, H., Wiersema, J.R., van der Meere, J.J., Thompson, M., ije/dyt261.
Sonuga-Barke, E., 2013. ADHD performance reflects inefficient but not impulsive Ratcliff, R., McKoon, G., 2008. The diffusion decision model: theory and data for
information processing: a diffusion model analysis. Neuropsychol- two-choice decision tasks. Neural Comput. 20, 873–922. http://dx.doi.org/10.1162/
ogy 27, 193–200. http://dx.doi.org/10.1037/a0031533. neco.2008.12-06-420.
Meyer, J.S., Quenzer, L.F., 2005. Psychopharmacology: Drugs, the Brain, and Behav- Ratcliff, R., Cherian, A., Cherian, A., Segraves, M., Segraves, M., 2003. A comparison
RE
ior, first edition Sinauer Associates, Publishers, Sunderland, MA. (555 p). of macaque behavior and superior colliculus neuronal activity to predictions from
Montague, P.R., Dayan, P., Sejnowski, T.J., 1996. A framework for mesencephalic models of two-Choice decisions. J. Neurophysiol. 90, 1392–1407. http://dx.doi.
dopamine systems based on predictive Hebbian learning. J. Neu- org/10.1152/jn.01049.2002.
rosci. 16, 1936–1947. Ratcliff, R., 1978. A theory of memory retrieval. Psychol. Rev. 85, http://dx.doi.org/
Montague, P.R., Hyman, S.E., Cohen, J.D., 2004. Computational roles for dopamine in 10.1037/0033-295X.85.2.59. 59–50.
behavioural control. Nature 431, 760–767. http://dx.doi.org/10.1038/nature03015. Rausch, V.H., Bauch, E.M., Bunzeck, N., 2014. White noise improves learning by
Montague, P.R., Dolan, R.J., Friston, K.J., Dayan, P., 2012. Computational psychiatry. modulating activity in dopaminergic midbrain regions and right superior temporal
R
Trends Cogn. Sci. 16, 72–80. http://dx.doi.org/10.1016/j.tics.2011.11.018. sulcus. J. Cogn. Neurosci. 26, 1469–1480. http://dx.doi.org/10.1162/jocn_a_
Moss, F., Ward, L.M., Sannita, W.G., 2004. Stochastic resonance and sensory informa- 00537.
tion processing: a tutorial and review of application. Clin. Neurophys- Rescorla, R.A., Wagner, A.R., 1972. A theory of Pavlovian conditioning: Variations in
iol. 115, 267–281. http://dx.doi.org/10.1016/j.clinph.2003.09.014. the effectiveness of reinforcement and nonreinforcement, in: Black, A.H., Prokasy,
CO
Mowinckel, A.M., Pedersen, M.L., Eilertsen, E., Biele, G., 2015. A meta-analysis of W.F., (Eds.), Classical Conditioning II: Current Research and Theory. New York
decision-making and attention in adults with ADHD. J. Atten. Dis- 64–99.
ord. 19, 355–367. http://dx.doi.org/10.1177/1087054714558872. Reynolds, J.N.J., Wickens, J.R., 2002. Dopamine-dependent plasticity of corticostriatal
Mulder, M.J., Bos, D., Weusten, J.M.H., van Belle, J., van Dijk, S.C., Simen, P., van synapses. Neural Netw. 15, 507–521.
Engeland, H., Durston, S., 2010. Basic impairments in regulating the speed-Accu- Rhodes, S.M., Coghill, D.R., Matthews, K., 2004. Methylphenidate restores visual
racy tradeoff predict symptoms of attention-Deficit/Hyperactivity disorder. Biol. memory, but not working memory function in attention deficit-hyperkinetic disor-
Psychiatry 68, 1114–1119. http://dx.doi.org/10.1016/j.biopsych.2010.07.031. der. Psychopharmacology (Berl.) http://dx.doi.org/10.1007/s00213-004-1833-7.
Mulder, M.J., van Maanen, L., Forstmann, B.U., 2014. Perceptual decision neuro- Rhodes, S.M., Coghill, D.R., Matthews, K., 2006. Acute neuropsychological effects of
sciences −A model-based review. Neuroscience 277, 872–884. http://dx.doi.org/ methylphenidate in stimulant drug-naïve boys with ADHD II–broader executive
UN
10.1016/j.neuroscience.2014.07.031. and non-executive domains. J. Child Psychol. Psychiatry 47, 1184–1194. http://dx.
Nair-Roberts, R.G., Chatelain-Badie, S.D., Benson, E., White-Cooper, H., Bolam, J.P., doi.org/10.1111/j. 1469-7610.2006.01633. x.
Ungless, M.A., 2008. Stereological estimates of dopaminergic, GABAergic and Rice
glutamatergic neurons in the ventral tegmental area, substantia nigra and and
retrorubral field in the rat. Neuroscience 152, 1024–1031. http://dx.doi.org/10. Cragg,
1016/j.neuroscience.2008.01.046.
Nicola, S.M., Hopf, F.W., Hjelmstad, G.O., 2004. Contrast enhancement: a physiologi- 2008. M.E. Rice, S.J. Cragg, Dopamine spillover after quantal release: rethinking
cal effect of striatal dopamine?. Cell Tissue Res. 318, 93–106. http://dx.doi.org/10. dopamine transmission in the nigrostriatal pathway, Brain Res. Rev. 58 (2008)
1007/s00441-004-0929-z. 303–313, http://dx.doi.org/10.1016/j.brainresrev.2008.02.004.
Nigg, J.T., 2013. Attention-deficit/hyperactivity disorder and adverse health outcomes. Rinne, U.K., Laihinen, A., Rinne, J.O., Någren, K., Bergman, J., Ruotsalainen, U.,
Clin. Psychol. Rev. 33, 215–228. http://dx.doi.org/10.1016/j.cpr.2012.11.005.
1990. Positron emission tomography demonstrates dopamine D2 receptor super-
Niv, Y., Schoenbaum, G., 2008. Dialogues on prediction errors. Trends Cogn.
sensitivity in the striatum of patients with early Parkinson's disease. Mov. Dis-
Sci. 12, 265–272. http://dx.doi.org/10.1016/j.tics.2008.03.006.
ord. 5, 55–59. http://dx.doi.org/10.1002/mds.870050114.
Robbins, T.W., Arnsten, A.F.T., 2009. The neuropsychopharmacology of fronto-exec- Seeman, P., Madras, B., 2002. Methylphenidate elevates resting dopamine which low-
utive function: monoaminergic modulation−PubMed−NCBI. Annu. Rev. Neu- ers the impulse-triggered release of dopamine: a hypothesis. Behav. Brain
rosci. 32, 267–287. http://dx.doi.org/10.1146/annurev.neuro.051508.135535. Res. 130, 79–83.
Roesch, M.R., Calu, D.J., Schoenbaum, G., 2007. Dopamine neurons encode the better Sergeant, J., 2000. The cognitive-energetic model: an empirical approach to Atten-
option in rats deciding between differently delayed or sized rewards. Nat. Neu- tion-Deficit Hyperactivity Disorder. Neurosci. Biobehav. Rev. 24, 7–12.
rosci. 10, 1615–1624. http://dx.doi.org/10.1038/nn2013. Sergeant, J.A., 2005. Modeling attention-deficit/hyperactivity disorder: a critical ap-
Ruskin, D.N., Bergstrom, D.A., Shenker, A., Freeman, L.E., Baek, D., Walters, J.R., praisal of the cognitive-energetic model. Biol. Psychiatry 57, 1248–1255. http://dx.
F
2001. Drugs used in the treatment of attention-deficit/hyperactivity disorder affect doi.org/10.1016/j.biopsych.2004.09.010.
postsynaptic firing rate and oscillation without preferential dopamine autoreceptor Shadlen, M.N., Newsome, W.T., 2001. Neural basis of a perceptual decision in the
action. Biol. Psychiatry 49, 340–350. parietal cortex (Area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936.
OO
Russell, V.A., Wiggins, T.M., 2000. Increased glutamate-Stimulated norepinephrine http://dx.doi.org/10.1038/84023.
release from prefrontal cortex slices of spontaneously hypertensive rats. Metab. Sikstrom, S., Söderlund, G., 2007. Stimulus-dependent dopamine release in atten-
Brain Dis. 15, 297–304. http://dx.doi.org/10.1023/A: 1011175225512. tion-deficit/hyperactivity disorder. Psychol. Rev. 114, 1047–1075. http://dx.doi.
Russell, V., de Villiers, A., Sagvolden, T., Lamm, M., Taljaard, J., 1995. Altered org/10.1037/0033-295X.114.4.1047.
dopaminergic function in the prefrontal cortex, nucleus accumbens and cau- Silvetti, M., Wiersema, J.R., Sonuga-Barke, E., Verguts, T., 2013. Deficient reinforce-
date-putamen of an animal model of attention-deficit hyperactivity disorder-the ment learning in medial frontal cortex as a model of dopamine-related motiva-
spontaneously hypertensive rat. Brain Res. 676, 343–351. tional deficits in ADHD. Neural Netw. 46, 199–209. http://dx.doi.org/10.1016/j.
Söderlund, G., Sikstrom, S., Smart, A., 2007. Listen to the noise: noise is beneficial for neunet.2013.05.008.
cognitive performance in ADHD. J. Child Psychol. Psyc. 48, 840–847. http://dx. Smith, P.L., Ratcliff, R., 2004. Psychology and neurobiology of simple decisions.
PR
doi.org/10.1111/j. 1469-7610.2007.01749. x. Trends Neurosci. 27, 161–168. http://dx.doi.org/10.1016/j.tins.2004.01.006.
Söderlund, G.B.W., Sikstrom, S., Loftesnes, J.M., Sonuga-Barke, E.J., 2010. The ef- Solanto, M.V., Conners, C.K., 1982. A dose-Response and time-Action analysis of au-
fects of background white noise on memory performance in inattentive school tonomic and behavioral-Effects of methylphenidate in attention deficit disorder
children. Behav. Brain Funct. 6, http://dx.doi.org/10.1186/1744-9081-6-55. with hyperactivity. Psychophysiology 19, 658–667. http://dx.doi.org/10.1111/j.
Söderlund, G.B.W., Eckernäs, D., Holmblad, O., Bergquist, F., 2015. Acoustic noise 1469-8986.1982. tb02519. x/pdf.
improves motor learning in spontaneously hypertensive rats, a rat model of atten- Solanto, M.V., Abikoff, H., Sonuga-Barke, E., Schachar, R., Logan, G.D., Wigal, T.,
tion deficit hyperactivity disorder. Behav. Brain Res. 280, 84–91. http://dx.doi.org/ Hechtman, L., Hinshaw, S., Turkel, E., 2001. The ecological validity of delay
10.1016/j.bbr.2014.11.032. aversion and response inhibition as measures of impulsivity in AD/HD: a supple-
Sagvolden, T., Aase, H., Zeiner, P., Berger, D., 1998. Altered reinforcement mecha- ment to the NIMH multimodal treatment study of AD/HD. J. Abnorm. Child Psy-
nisms in attention-deficit/hyperactivity disorder. Behav. Brain Res. 94, 61–71. chol. 29, 215–228. http://dx.doi.org/10.1023/A: 1010329714819.
ED
http://dx.doi.org/10.1016/S0166-4328(97)00170-8. Solanto, M.V., Arnsten, A.F.T., Castellanos, F.X., 2001. Stimulant Drugs and ADHD:
Sagvolden, T., Johansen, E.B., Aase, H., Russell, V.A., 2005. A dynamic developmen- Basic and Clinical Neuroscience, Oxford University Press, New York.
tal theory of attention-deficit/hyperactivity disorder (ADHD) predominantly hy- Solanto, M.V., 2002. Dopamine dysfunction in AD/HD: integrating clinical and basic
peractive/impulsive and combined subtypes. Behav. Brain Sci. 28, 397–419. http:// neuroscience research. Behav. Brain Res. 130, 65–71.
dx.doi.org/10.1017/S0140525x05000075. discussion 419–68. Sonuga-Barke, E.J.S., Fairchild, G., 2012. Neuroeconomics of attention-deficit/hyper-
Sagvolden, T., 2000. Behavioral validation of the spontaneously hypertensive rat activity disorder: differential influences of medial, dorsal, and ventral prefrontal
(SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/HD). brain networks on suboptimal decision making?. Biol. Psychiatry 72, 126–133.
Neurosci. Biobehav. Rev. 24, 31–39. http://dx.doi.org/10.1016/j.biopsych.2012.04.004.
CT
Salum, G.A., Sergeant, J., Sonuga-Barke, E., Vandekerckhove, J., Gadelha, A., Pan, Sonuga-Barke, E.J.S., 2003. The dual pathway model of AD/HD: an elaboration of
P.M., Moriyama, T.S., Graeff-Martins, A.S., de Alvarenga, P.G., do Rosário, neuro-developmental characteristics. Neurosci. Biobehav. Rev. 27, 593–604. http:
M.C., Manfro, G.G., Polanczyk, G., Rohde, L.A.P., 2014. Specificity of basic in- //dx.doi.org/10.1016/j.neubiorev.2003.08.005.
formation processing and inhibitory control in attention deficit hyperactivity disor- Spencer, T.J., Brown, A., Seidman, L.J., Valera, E.M., Makris, N., Lomedico, A.,
der. Psychol. Med. 44, 617–631. http://dx.doi.org/10.1017/s0033291713000639. Faraone, S.V., Biederman, J., 2013. Effect of psychostimulants on brain structure
Salum, G.A., Sonuga-Barke, E., Sergeant, J., Vandekerckhove, J., Gadelha, A., and function in ADHD: a qualitative literature review of magnetic resonance imag-
Moriyama, T.S., Graeff-Martins, A.S., Manfro, G.G., Polanczyk, G., Rohde, ing-based neuroimaging studies. J. Clin. Psychiatry 74, 902–917. http://dx.doi.org/
10.4088/JCP.12r08287.
RE
L.A.P., 2014. Mechanisms underpinning inattention and hyperactivity: neurocog-

nitive support for ADHD dimensionality. Psychol. Med. 44, 3189–3201. http://dx. Stanford, C., Tannock, R., 2012. Behavioral Neuroscience of Attention Deficit Hyper-
doi.org/10.1017/S0033291714000919. activity Disorder and Its Treatment. Springer, Berlin, Heidelberg.
Scheres, A., Milham, M.P., Knutson, B., Castellanos, F.X., 2007. Ventral striatal hy- Suri, R.E., Schultz, W., 2001. Temporal difference model reproduces anticipatory
poresponsiveness during reward anticipation in attention-deficit/hyperactivity dis- neural activity. Neural Comput. 13, 841–862. http://dx.doi.org/10.1162/
order. Biol. Psychiatry 61, 720–724. http://dx.doi.org/10.1016/j.biopsych.2006.04. 089976601300014376.
042. Sutton, R.S., Barto, A.G., 1998. Reinforcement Learning: an Introduction. MIT Press,
Schiffer, W.K., Volkow, N.D., Fowler, J.S., Alexoff, D.L., Logan, J., Dewey, S.L., Cambridge, MA.
R
2006. Therapeutic doses of amphetamine or methylphenidate differentially in- Swanson, J., Baler, R.D., Volkow, N.D., 2010. Understanding the effects of stimulant
crease synaptic and extracellular dopamine. Synapse 59, 243–251. http://dx.doi. medications on cognition in individuals with attention-Deficit hyperactivity disor-
org/10.1002/syn.20235. der: a decade of progress. Neuropsychopharmacology 36, 207–226. http://dx.doi.
Schonberg, T., Daw, N.D., Joel, D., O'Doherty, J.P., 2007. Reinforcement learning sig- org/10.1038/npp.2010.160.
CO
nals in the human striatum distinguish learners from nonlearners during re- Szalma, J.L., Hancock, P.A., 2011. Noise effects on human performance: a meta-ana-
ward-Based decision making. J. Neurosci. 27, 12860–12867. http://dx.doi.org/10. lytic synthesis. Psychol. Bull. 137, 682–707. http://dx.doi.org/10.1037/a0023987.
1523/JNEUROSCI. 2496-07.2007. Tarpy, R.M., Sawabini, F.L., 1974. Reinforcement delay − selective review of last
Schultz, W., Apicella, P., Scarnati, E., Ljungberg, T., 1992. Neuronal-Activity in mon- decade. Psychol. Bull. 81, 984–997. http://dx.doi.org/10.1037/h0037428.
key ventral striatum related to the expectation of reward. J. Neu- Tobler, P.N., Fiorillo, C.D., Schultz, W., 2005. Adaptive coding of reward value by
rosci. 12, 4595–4610. dopamine neurons. Science 307, 1642–1645. http://dx.doi.org/10.1126/science.
Schultz, W., 2000. Multiple reward signals in the brain. Nat. Rev. Neu- 1105370.
rosci. 1, 199–207. http://dx.doi.org/10.1038/35044563. Tripp, G., Alsop, B., 1999. Sensitivity to reward frequency in boys with attention
Schultz, W., 2002. Getting formal with dopamine and reward. Neuron 36, 241–263. deficit hyperactivity disorder. J. Clin. Child Psychol. 28, 366–375.
UN
http://dx.doi.org/10.1016/S0896-6273(02)00967-4. Tripp, G., Alsop, B., 2001. Sensitivity to reward delay in children with attention deficit
Schultz, W., 2007. Multiple dopamine functions at different time courses. Annu. Rev. hyperactivity disorder (ADHD). J. Child Psychol. Psyc. 42, 691–698. http://dx.doi.
Neurosci. 30, 259–288. http://dx.doi.org/10.1146/annurev.neuro.28.061604. org/10.1111/1469-7610.00764.
135722. Tripp, G., Wickens, J.R., 2008. Research review: dopamine transfer deficit: a neurobi-
Schultz, W., 2007. Behavioral dopamine signals. Trends Neurosci. 30, 203–210. http:// ological theory of altered reinforcement mechanisms in ADHD. J. Child Psychol.
dx.doi.org/10.1016/j.tins.2007.03.007. Psyc. 49, 691–704. http://dx.doi.org/10.1111/j. 1469-7610.2007.01851. x.
Schultz, W., 2007. Behavioral dopamine signals. Trends Neurosci. 30, 203–210. http:// Tripp, G., Wickens, J.R., 2009. Neurobiology of ADHD. Neuropharmacol-
dx.doi.org/10.1016/j.tins.2007.03.007. ogy 57, 579–589. http://dx.doi.org/10.1016/j.neuropharm.2009.07.026.
Schultz, W., 2010. Dopamine signals for reward value and risk: basic and recent data. Tripp, G., Wickens, J.R., 2012. Reinforcement, dopamine and rodent models in drug
Behav Brain Funct 6, 24. http://dx.doi.org/10.1186/1744-9081-6-24. development for ADHD. Neurotherapeutics 9, 622–634. http://dx.doi.org/10.1007/
Schultz, W., 2013. Updating dopamine reward signals. Curr. Opin. Neuro- s13311-012-0132-y.
biol. 23, 229–238. http://dx.doi.org/10.1016/j.conb.2012.11.012. Tsai, H.C., Zhang, F., Adamantidis, A., Stuber, G.D., Bonci, A., de Lecea, L., Deis-
seroth, K., 2009. Phasic firing in dopaminergic neurons is sufficient for behavioral
conditioning. Science 324, 1080–1084. http://dx.doi.org/10.1126/science.1168878.
Usher, M., McClelland, J.L., 2001. The time course of perceptual choice: the leaky, Wiecki, T.V., Sofer, I., Frank, M.J., 2013. HDDM: hierarchical bayesian estimation of
competing accumulator model. Psychol. Rev. 108, 550–592. http://dx.doi.org/10. the drift-Diffusion model in python. Front. Neuroinf. 7, 14. http://dx.doi.org/10.
1037/0033-295X.108.3.550. 3389/fninf.2013.00014.
van der Kooij, M.A., Glennon, J.C., 2007. Animal models concerning the role of Wiersema, R., van der Meere, J., Roeyers, H., Van Coster, R., Baeyens, D., 2006.
dopamine in attention-deficit hyperactivity disorder. Neurosci. Biobehav. Event rate and event-related potentials in ADHD. J. Child Psychol. Psychia-
Rev. 31, 597–618. http://dx.doi.org/10.1016/j.neubiorev.2006.12.002. try 47, 560–567. http://dx.doi.org/10.1111/j. 1469-7610.2005.01592. x.
Vijayraghavan, S., Wang, M., Birnbaum, S.G., Williams, G.V., Arnsten, A.F.T., 2007. Wilens, T.E., 2008. Effects of methylphenidate on the catecholaminergic system in
F
Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in work- attention-Deficit/Hyperactivity disorder. J. Clin. Psychopharmacol. 28, S46–S53.
ing memory. Nat. Neurosci. 10, 376–384. http://dx.doi.org/10.1038/nn1846. http://dx.doi.org/10.1097/JCP.0b013e318173312f.
Volkow, N.D., Wang, G., Fowler, J.S., Logan, J., Gerasimov, M., Maynard, L., Ding, Willcutt, E.G., Doyle, A.E., Nigg, J.T., Faraone, S.V., Pennington, B.F., 2005. Validity
OO
Y., Gatley, S.J., Gifford, A., Franceschi, D., 2001. Therapeutic doses of oral of the executive function theory of attention-Deficit/Hyperactivity disorder: a
methylphenidate significantly increase extracellular dopamine in the human brain. meta-Analytic review. Biol. Psychiatry 57, 1336–1346. http://dx.doi.org/10.1016/j.
J. Neurosci. 21, 1–5. biopsych.2005.02.006.
Volkow, N.D., Wang, G.-J., Kollins, S.H., Wigal, T.L., Newcorn, J.H., Telang, F., Willcutt, E.G., Sonuga-Barke, E.J.S., Nigg, J.T., Sergeant, J.A., 2008. Recent develop-
Fowler, J.S., Zhu, W., Logan, J., Ma, Y., Pradhan, K., Wong, C., Swanson, J.M., ments in neuropsychological models of childhood psychiatric disorders. In: Ba-
2009. Evaluating dopamine reward pathway in ADHD: clinical implications. naschewski, T., Rohde, L.A. (Eds.), Biological Child Psychiatry, Advances in Bio-
JAMA 302, 1084–1091. http://dx.doi.org/10.1001/jama.2009.1308. logical Psychiatry.. KARGER, Basel, pp. 195–226. http://dx.doi.org/10.1159/
Waelti, P., Dickinson, A., Schultz, W., 2001. Dopamine responses comply with basic 000118526.
assumptions of formal learning theory. Nature 412, 43–48. http://dx.doi.org/10. Williams, J., Dayan, P., 2005. Dopamine, learning, and impulsivity: a biological ac-
PR
1038/35083500. count of attention-deficit/hyperactivity disorder. J. Child Adolesc. Psychopharma-
Wagenmakers, E.-J., van der Maas, H.L.J., Grasman, R.P.P.P., 2007. An EZ-diffusion col. 15, 160–179. http://dx.doi.org/10.1089/cap.2005.15.160. (discussion 157-9).
model for response time and accuracy. Psychon. Bull. Rev. 14, 3–22. http://dx.doi. Winkel, J., van Maanen, L., Ratcliff, R., van der Schaaf, M.E., van Schouwenburg,
org/10.3758/BF03194023. M.R., Cools, R., Forstmann, B.U., 2012. Bromocriptine does not alter speed-accu-
CJCH, Watkins, Dayan, P., 1992. Q-learning. Mach Learn 8 (3–4), 279–292. http://dx. racy tradeoff. Front. Neurosci. 6, 126. http://dx.doi.org/10.3389/fnins.2012.00126.
doi.org/10.1007/BF00992698. Winstanley, C.A., Theobald, D.E.H., Cardinal, R.N., Robbins, T.W., 2004. Contrasting
Weigard, A., Huang-Pollock, C., 2014. A diffusion modeling approach to understand- roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neu-
ing contextual cueing effects in children with ADHD. J. Child Psychol. Psyc. http: rosci. 24, 4718–4722. http://dx.doi.org/10.1523/JNEUROSCI. 5606-03.2004.
//dx.doi.org/10.1111/jcpp.12250. (n/a-n/a). Zimmer, L., 2009. Positron emission tomography neuroimaging for a better under-
White, C.N., Ratcliff, R., Vasey, M.W., McKoon, G., 2010. Using diffusion models to standing of the biology of ADHD. Neuropharmacology 57, 601–607. http://dx.doi.
ED
understand clinical disorders. J. Math. Psychol. 54, 39–52. http://dx.doi.org/10. org/10.1016/j.neuropharm.2009.08.001.
1016/j.jmp.2010.01.004.
CT
R RE
CO
UN

Modelling ADHD Uncorrected Proof

Uploaded by

Copyright:

Available Formats

Modelling ADHD Uncorrected Proof

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Modelling ADHD Uncorrected Proof

Uploaded by

Copyright:

Available Formats

Neuroscience and Biobehavioral Reviews xxx (2016) xxx-xxx

Contents lists available at ScienceDirect

Neuroscience and Biobehavioral Reviews

Modelling ADHD: A review of ADHD theories through their predictions for

ARTICLE INFO ABSTRACT

© 2016 Published by Elsevier Ltd.

coeruleus (Berridge and Waterhouse, 2003). Axons from noradrener-

time (RT). Reduced accuracy can be caused either by fast respond-

2004). This is largely because models provide mechanistic explana-

3.1. Sequential sampling models of decision-making

Sequential sampling models are computational models used to

tical firing (Aston-Jones and Cohen, 2005; Durstewitz and Seamans,

Change in model parameters relative to typically developing controls

Drift-Diffusion Model PERL model

Basal Ganglia ↓/↑ ↓ ↓ ↑ ↓

4.2.4.2. Boundary separation

4.2.5.1. Learning rate 4.3.2. Theory description

should also be inversely correlated with variability in RTs since both

4.2.6. Predictions of developmental effects from altered

4.3. The dynamic developmental theory

4.3.1. Biological presumptions

selective reinforcement of response chains with short inter-response

Building on a study by Cardinal et al. (2000), the authors also propose

Grace’s tonic/phasic model (Grace, 2000, 1991). There is also some

is generally used to designate an overall psychological and physio-

should be experimentally controlled for.

ADHD (Sikstrom and Söderlund, 2007). This should manifest as a

ent arousing properties of task conditions (Sikstrom and Söderlund,

Table 2 Null findings could − at least partially − be due to small effects of

centive structure. Thus it might be viewed more as a useful indicator

ries reviewed herein.

minutes to hours scale when clinically efficient plasma levels of stim-

functioning, temporal discounting, and sense of time in adolescents with attention

Johansen, E.B., Sagvolden, T., 2004. Response disinhibition may be explained as an

134–157. measurement and mechanisms of a proposed trans-diagnostic phenotype. J. Child

L.A.P., 2014. Mechanisms underpinning inattention and hyperactivity: neurocog-

You might also like