What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminer... more What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning?
Abstract: Planning and executing volitional actions in the face of conflicting habitual responses... more Abstract: Planning and executing volitional actions in the face of conflicting habitual responses is a critical aspect of human behavior. At the core of the interplay between these two control systems lies an override mechanism that can suppress the habitual action selection process and allow executive control to take over.
Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline ... more Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit/hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in'Go'learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations.
Abstract What are the genetic and neural components that support adaptive learning from positive ... more Abstract What are the genetic and neural components that support adaptive learning from positive and negative outcomes? Here, we show with genetic analyses that three independent dopaminergic mechanisms contribute to reward and avoidance learning in humans. A polymorphism in the DARPP-32 gene, associated with striatal dopamine function, predicted relatively better probabilistic reward learning.
Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both fron... more Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA “overdose.” These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system.
BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate stri... more BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate striatal-cortical pathways. We investigated whether fronto-striatal dysfunction in schizophrenia (SZ) is characterized by selective impairment in either reward-(Go) or punishment-driven (NoGo) learning. METHODS: We administered two versions of a probabilistic selection task to 40 schizophrenia patients and 31 control subjects, using difficult to verbalize stimuli (experiment 1) and nameable objects (experiment 2).
Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? ... more Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error.
We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work t... more We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning.
Investigations into action monitoring have consistently detailed a frontocentral voltage deflecti... more Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested.
Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have... more Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have been implicated to underlie obtrusive thoughts and repetitive behaviors in Obsessive-Compulsive Disorder (OCD). Larger Error-Related Negativities (ERNs) in OCD patients during simple flanker tasks have been proposed to reflect an amplified error signal in these hyperactive circuits.
Computational models of the basal ganglia have matured and received increasing attention over the... more Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.
Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, ... more Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically.
Over the last decade and a half, reinforcement learning models have fostered an increasingly soph... more Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits.
As noted in the main text, the CEVR condition was included to compare with CEV as a measure of pr... more As noted in the main text, the CEVR condition was included to compare with CEV as a measure of probability-magnitude bias (PM-bias= CEVR-CEV). We found PM-bias to be positive across all groups (p<. 001; Figure S1), as participants avoided responding early in CEVR due to the low reward probability at that time, consistent with loss-aversion (1).
Abstract To examine how stress affects cognitive functioning, individual differences in trait vul... more Abstract To examine how stress affects cognitive functioning, individual differences in trait vulnerability (punishment sensitivity) and state reactivity (negative affect) to social evaluative threat were examined during concurrent reinforcement learning. Lower trait-level punishment sensitivity predicted better reward learning and poorer punishment learning; the opposite pattern was found in more punishment sensitive individuals.
Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning,... more Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning, previous theoretical and empirical accounts have suggested that increased uncertainty may precede the decision to explore an alternative option. To date, the neural mechanisms that support the strategic application of uncertainty-driven exploration remain underspecified. In this study, electroencephalography (EEG) was used to assess trial-to-trial dynamics relevant to exploration and exploitation.
Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report le... more Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report levels of emotional experience that are similar to controls (CN) when asked to view a single evocative stimulus and make an absolute judgment of stimulus “value.” However, value is rarely assigned in absolute terms in real-life situations, where one alternative or experience is often evaluated alongside others, and value judgments are made in relative terms.
Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of in... more Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia.
Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervati... more Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade.
What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminer... more What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning?
Abstract: Planning and executing volitional actions in the face of conflicting habitual responses... more Abstract: Planning and executing volitional actions in the face of conflicting habitual responses is a critical aspect of human behavior. At the core of the interplay between these two control systems lies an override mechanism that can suppress the habitual action selection process and allow executive control to take over.
Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline ... more Abstract We test our neurocomputational model of fronto-striatal dopamine (DA) and noradrenaline (NA) function for understanding cognitive and motivational deficits in attention deficit/hyperactivity disorder (ADHD). Our model predicts that low striatal DA levels in ADHD should lead to deficits in'Go'learning from positive reinforcement, which should be alleviated by stimulant medications, as observed with DA manipulations in other populations.
Abstract What are the genetic and neural components that support adaptive learning from positive ... more Abstract What are the genetic and neural components that support adaptive learning from positive and negative outcomes? Here, we show with genetic analyses that three independent dopaminergic mechanisms contribute to reward and avoidance learning in humans. A polymorphism in the DARPP-32 gene, associated with striatal dopamine function, predicted relatively better probabilistic reward learning.
Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both fron... more Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA “overdose.” These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system.
BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate stri... more BACKGROUND: Rewards and punishments may make distinct contributions to learning via separate striatal-cortical pathways. We investigated whether fronto-striatal dysfunction in schizophrenia (SZ) is characterized by selective impairment in either reward-(Go) or punishment-driven (NoGo) learning. METHODS: We administered two versions of a probabilistic selection task to 40 schizophrenia patients and 31 control subjects, using difficult to verbalize stimuli (experiment 1) and nameable objects (experiment 2).
Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? ... more Abstract To what extent do we learn from the positive versus negative outcomes of our decisions? The neuromodulator dopamine plays a key role in these reinforcement learning processes. Patients with Parkinson's disease, who have depleted dopamine in the basal ganglia, are impaired in tasks that require learning from trial and error.
We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work t... more We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning.
Investigations into action monitoring have consistently detailed a frontocentral voltage deflecti... more Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested.
Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have... more Abstract Hyperactive cortico-striatal circuits including the Anterior Cingulate Cortex (ACC) have been implicated to underlie obtrusive thoughts and repetitive behaviors in Obsessive-Compulsive Disorder (OCD). Larger Error-Related Negativities (ERNs) in OCD patients during simple flanker tasks have been proposed to reflect an amplified error signal in these hyperactive circuits.
Computational models of the basal ganglia have matured and received increasing attention over the... more Computational models of the basal ganglia have matured and received increasing attention over the last decade. This article reviews some of the theoretical advances offered by these models, focusing on motor and cognitive action selection, learning, and the interaction between multiple corticostriatal circuits in selection and learning.
Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, ... more Abstract Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically.
Over the last decade and a half, reinforcement learning models have fostered an increasingly soph... more Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits.
As noted in the main text, the CEVR condition was included to compare with CEV as a measure of pr... more As noted in the main text, the CEVR condition was included to compare with CEV as a measure of probability-magnitude bias (PM-bias= CEVR-CEV). We found PM-bias to be positive across all groups (p<. 001; Figure S1), as participants avoided responding early in CEVR due to the low reward probability at that time, consistent with loss-aversion (1).
Abstract To examine how stress affects cognitive functioning, individual differences in trait vul... more Abstract To examine how stress affects cognitive functioning, individual differences in trait vulnerability (punishment sensitivity) and state reactivity (negative affect) to social evaluative threat were examined during concurrent reinforcement learning. Lower trait-level punishment sensitivity predicted better reward learning and poorer punishment learning; the opposite pattern was found in more punishment sensitive individuals.
Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning,... more Abstract In order to understand the exploitation/exploration trade-off in reinforcement learning, previous theoretical and empirical accounts have suggested that increased uncertainty may precede the decision to explore an alternative option. To date, the neural mechanisms that support the strategic application of uncertainty-driven exploration remain underspecified. In this study, electroencephalography (EEG) was used to assess trial-to-trial dynamics relevant to exploration and exploitation.
Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report le... more Abstract Previous studies have typically found that individuals with schizophrenia (SZ) report levels of emotional experience that are similar to controls (CN) when asked to view a single evocative stimulus and make an absolute judgment of stimulus “value.” However, value is rarely assigned in absolute terms in real-life situations, where one alternative or experience is often evaluated alongside others, and value judgments are made in relative terms.
Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of in... more Abstract Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia.
Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervati... more Summary Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade.
Uploads
Papers by Michael J Frank