poster

Joy, distress, hope, and fear in reinforcement learning

Authors:

Joost Broekens,

Catholijn JonkerAuthors Info & Claims

AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems

Pages 1615 - 1616

Published: 05 May 2014 Publication History

Abstract

In this paper we present a mapping between joy, distress, hope and fear, and Reinforcement Learning primitives. Joy / distress is a signal that is derived from the RL update signal, while hope/fear is derived from the utility of the current state. Agent-based simulation experiments replicate psychological and behavioral dynamics of emotion including: joy and distress reactions that develop prior to hope and fear; fear extinction; habituation of joy; and, task randomness that increases the intensity of joy and distress. This work distinguishes itself by assessing the dynamics of emotion in an adaptive agent framework - coupling it to the literature on habituation, development, and extinction.

References

[1]

Joost Broekens, Stacy Marsella, and Tibor Bosse. Challenges in computational modeling of affective processes. IEEE Transactions on Affective Computing, 4(3), 2013.

Digital Library

[2]

L. Canamero. Emotion understanding from the perspective of autonomous robots research. Neural networks, 18(4):445--455, 2005.

Digital Library

[3]

A. R. Damasio. Descartes' Error: emotion reason and the human brain. Penguin Putnam, 1996.

[4]

Peter Dayan and Bernard W. Balleine. Reward, motivation, and reinforcement learning. Neuron, 36(2):285--298, 2002.

[5]

Magy Seif El-Nasr, John Yen, and Thomas R Ioerger. Flame: fuzzy logic adaptive model of emotions. Autonomous Agents and Multi-agent systems, 3(3):219--257, 2000.

Digital Library

[6]

N. H. Frijda. Emotions and action, page 158--173. Cambridge University Press, 2004.

[7]

N.H. Frijda, P. Kuipers, and E. Ter Schure. Relations among emotion, appraisal, and emotional action readiness. Journal of Personality and Social Psychology, 57(2):212, 1989.

[8]

Elmer Jacobs, Joost Broekens, and Catholijn Jonker. Emergent dynamics of joy, distress, hope and fear in reinforcement learning agents. In Adaptive Learning Agents workshop at AAMAS2014, 2014.

[9]

Stacy Marsella, Jonathan Gratch, and Paolo Petta. Computational models of emotion. K. r. Scherer, t. BÃ¤nziger and e. roesch (eds.), A blueprint for affective computing, pages 21--45, 2010.

[10]

K. M. Myers and M. Davis. Mechanisms of fear extinction. Mol Psychiatry, 12(2):120--150, 2006.

[11]

John P. O'Doherty. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Current opinion in neurobiology, 14(6):769--776, 2004.

[12]

Andrew Ortony, Gerald L. Clore, and Allan Collins. The Cognitive Structure of Emotions. Cambridge University Press, 1988.

[13]

Edmund T. Rolls. Precis of the brain and emotion. Behavioral and Brain Sciences, 20:177--234, 2000.

[14]

Edmund T. Rolls and Fabian Grabenhorst. The orbitofrontal cortex and beyond: From affect to decision-making. Progress in Neurobiology, 86(3):216--244, 2008.

[15]

K.R. Scherer. Appraisal considered as a process of multilevel sequential checking. Appraisal processes in emotion: Theory, methods, research, 92:120, 2001.

[16]

N. Schweighofer and K. Doya. Meta-learning in reinforcement learning. Neural Networks, 16(1):5--9, 2003.

Digital Library

[17]

Pedro Sequeira, FranciscoS Melo, and Ana Paiva. Emotion-Based Intrinsic Motivation for Reinforcement Learning Agents, volume 6974 of Lecture Notes in Computer Science, chapter 36, pages 326--336. Springer Berlin Heidelberg, 2011.

Digital Library

[18]

JC Sprott. Dynamical models of happiness. Nonlinear Dynamics, Psychology, and Life Sciences, 9(1):23--36, 2005.

[19]

L Alan Sroufe. Emotional development: The organization of emotional life in the early years. Cambridge University Press, 1997.

[20]

Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. Cambridge Univ Press, 1998.

Digital Library

[21]

Ruut Veenhoven. Is happiness relative? Social Indicators Research, 24(1):1--34, 1991.

Cited By

Daswani MLeike J(2015)A Definition of Happiness for Reinforcement Learning AgentsProceedings of the 8th International Conference on Artificial General Intelligence - Volume 920510.1007/978-3-319-21365-1_24(231-240)Online publication date: 22-Jul-2015
https://dl.acm.org/doi/10.1007/978-3-319-21365-1_24

Index Terms

Joy, distress, hope, and fear in reinforcement learning
1. Computing methodologies
  1. Machine learning

Recommendations

A reinforcement learning model of joy, distress, hope and fear
Adaptive Learning Agents, Part 3

In this paper we computationally study the relation between adaptive behaviour and emotion. Using the reinforcement learning framework, we propose that learned state utility, <inline-formula><inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" ...
Affective computing with primary and secondary emotions in a virtual human

We introduce the WASABI ([W]ASABI [A]ffect [S]imulation for [A]gents with [B]elievable [I]nteractivity) Affect Simulation Architecture, in which a virtual human's cognitive reasoning capabilities are combined with simulated embodiment to achieve the ...
Towards a Dynamic Model for the Prediction of Emotion Intensity from Peripheral Physiological Signals
HCI International 2022 - Late Breaking Papers. Multimodality in Advanced Interaction Environments
Abstract
Natural human-system interaction can facilitate the acceptance of technological systems. The ability of emotion recognition can hereby provide a significant contribution. Surprisingly, the field of emotion recognition is dominated by static ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems

May 2014

1774 pages

ISBN:9781450327381

General Chairs:
Ana Bazzan
UFRGS, Brazil
,
Michael Huhns
University of South Carolina, USA
,
Program Chairs:
Alessio Lomuscio
Imperial College London, UK
,
Paul Scerri
Carnegie Mellon University, USA

Sponsors

IFAAMAS

In-Cooperation

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 05 May 2014

Check for updates

Author Tags

Qualifiers

Poster

Conference

AAMAS '14

Sponsor:

AAMAS '14: International conference on Autonomous Agents and Multi-Agent Systems

May 5 - 9, 2014

Paris, France

Acceptance Rates

AAMAS '14 Paper Acceptance Rate 169 of 709 submissions, 24%;

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
128
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Daswani MLeike J(2015)A Definition of Happiness for Reinforcement Learning AgentsProceedings of the 8th International Conference on Artificial General Intelligence - Volume 920510.1007/978-3-319-21365-1_24(231-240)Online publication date: 22-Jul-2015
https://dl.acm.org/doi/10.1007/978-3-319-21365-1_24

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents