A story of two streams: Reinforcement learning models from human behavior and neuropsychiatry

B Lin, G Cecchi, D Bouneffouf, J Reinen… - arXiv preprint arXiv …, 2019 - arxiv.org
arXiv preprint arXiv:1906.11286, 2019arxiv.org
Drawing an inspiration from behavioral studies of human decision making, we propose here
a more general and flexible parametric framework for reinforcement learning that extends
standard Q-learning to a two-stream model for processing positive and negative rewards,
and allows to incorporate a wide range of reward-processing biases--an important
component of human decision making which can help us better understand a wide spectrum
of multi-agent interactions in complex real-world socioeconomic systems, as well as various …
Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.
arxiv.org