In inverse reinforcement learning an observer infers the reward distribution available for action... more In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.
The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain... more The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor). We found a degree of regional specialization within the SN: Whereas a region of ventromedial SN correlated with a temporal difference reward prediction error during appetitive Pavlovian learning, a dorsolateral area correlated instead with an aversive expected value signal in response to the most distal cue, and to a reward prediction error in response to the most proximal cue to the aversive outcome. Furthermore, participants' affective reactions to both the appetitive and aversive conditioned stimuli more than 1 year after the fMRI experiment was conducted correlated with activation in the ventromedial and dorsolateral SN obtained during the experiment, respectively. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts.
Human experience takes place in the line of mental time (MT) created through ‘self-projection’ of... more Human experience takes place in the line of mental time (MT) created through ‘self-projection’ of oneself to different time-points in the past or future. Here we manipulated self-projection in MT not only with respect to one’s life events but also with respect to one’s faces from different past and future time-points. Behavioural and event-related functional magnetic resonance imaging activity showed three independent effects characterized by (i) similarity between past recollection and future imagination, (ii) facilitation of judgements related to the future as compared with the past, and (iii) facilitation of judgements related to time-points distant from the present. These effects were found with respect to faces and events, and also suggest that brain mechanisms of MT are independent of whether actual life episodes have to be re-experienced or pre-experienced, recruiting a common cerebral network including the anteromedial temporal, posterior parietal, inferior frontal, temporo-parietal and insular cortices. These behavioural and neural data suggest that self-projection in time is a fundamental aspect of MT, relying on neural structures encoding memory, mental imagery and self.
In inverse reinforcement learning an observer infers the reward distribution available for action... more In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.
The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain... more The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor). We found a degree of regional specialization within the SN: Whereas a region of ventromedial SN correlated with a temporal difference reward prediction error during appetitive Pavlovian learning, a dorsolateral area correlated instead with an aversive expected value signal in response to the most distal cue, and to a reward prediction error in response to the most proximal cue to the aversive outcome. Furthermore, participants' affective reactions to both the appetitive and aversive conditioned stimuli more than 1 year after the fMRI experiment was conducted correlated with activation in the ventromedial and dorsolateral SN obtained during the experiment, respectively. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts.
Human experience takes place in the line of mental time (MT) created through ‘self-projection’ of... more Human experience takes place in the line of mental time (MT) created through ‘self-projection’ of oneself to different time-points in the past or future. Here we manipulated self-projection in MT not only with respect to one’s life events but also with respect to one’s faces from different past and future time-points. Behavioural and event-related functional magnetic resonance imaging activity showed three independent effects characterized by (i) similarity between past recollection and future imagination, (ii) facilitation of judgements related to the future as compared with the past, and (iii) facilitation of judgements related to time-points distant from the present. These effects were found with respect to faces and events, and also suggest that brain mechanisms of MT are independent of whether actual life episodes have to be re-experienced or pre-experienced, recruiting a common cerebral network including the anteromedial temporal, posterior parietal, inferior frontal, temporo-parietal and insular cortices. These behavioural and neural data suggest that self-projection in time is a fundamental aspect of MT, relying on neural structures encoding memory, mental imagery and self.
Uploads
Papers by Sven C.