extended-abstract

The Secret Language of Our Body: Affect and Personality Recognition Using Physiological Signals

Author:

Julia WacheAuthors Info & Claims

ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Pages 389 - 393

https://doi.org/10.1145/2663204.2666290

Published: 12 November 2014 Publication History

Abstract

We present a novel framework for decoding individuals? emotional state and personality traits based on physiological responses to affective movie clips. During watching 36 video clips we used measures of Electrocardiogram (ECG), Galvanic Skin Response (GSR), facial-Electroencephalogram (EEG) and facial emotional responses to decode i) the emotional state of partcipants and ii) their Big Five personality traits extending previous work that had connected either explicit (user ratings) with some implicit (physiological) affective responses or one of them with selected personality traits.

We make the first dataset comprising both affective and personality information publicly available for further research and we further explore different methods and implementations for automated emotion and personality detection for future applications.

References

[1]

Abadi, M. K., Kia, M., Subramanian, R., Avesani, P., and Sebe, N. Decoding Affect in Videos Employing the MEG Brain Signal. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), (2013), 1--6.

[2]

Abadi, M. K., Kia, S. M., Subramanian, R., Avesani, P., and Sebe, N. User-centric Affective Video Tagging from MEG and Peripheral Physiological Responses. 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, (2013), 582--587.

Digital Library

[3]

Adams, B., Dorai, C., and Venkatesh, S. Novel approach to determining tempo and dramatic story sections in motion pictures. Image Processing, 2000. Proceedings. 2000 International Conference on Vol. 2, IEEE., (2000), 283--286.

[4]

Argamon, S., Dhawle, S., Koppel, M., and Pennbaker, J. W. Lexical predictors of personality type. Interface and the Classification Society of North America, (2005).

[5]

Hanjalic, A. and Xu, L.-Q. Affective video content representation and modeling. IEEE Transactions on Multimedia 7, 1 (2005), 143--154.

Digital Library

[6]

Joho, H., Staiano, J., Sebe, N., and Jose, J.M. Looking at the Viewer: Analysing Facial Activities to Detect Personal Highlights of Multimedia Contents. Multimedia Tools and Applications 51, 2 (2011), 505--523.

Digital Library

[7]

Kehoe, E. G., Toomey, J. M., Balsters, J. H., and Bokde, A. L. W. Personality modulates the effects of emotional arousal and valence on brain activation. Social cognitive and affective neuroscience 7, 7 (2012), 858--70.

[8]

Kim, J. and André, E. Emotion recognition based on physiological changes in music listening. IEEE transactions on pattern analysis and machine intelligence 30, 12 (2008), 2067--83.

Digital Library

[9]

Kim, K. H., Bang, S. W., and Kim, S. R. Emotion recognition system using short-term monitoring of physiological signals. Medical & biological engineering & computing 42, 3 (2004), 419--27.

[10]

Koelstra, S., Mühl, C., Soleymani, M., et al. DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE Transactions on Affective Computing 3, 1 (2012), 18--31.

Digital Library

[11]

Lisetti, C.L. and Nasoz, F. Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP Journal on Advances in Signal Processing 2004, 11 (2004), 1672--1687.

Digital Library

[12]

McCrae, R. R. and John, O. P. An introduction to the five-factor model and its applications. Journal of personality 60, 2 (1992), 175--215.

[13]

Ng, W. Clarifying the relation between neuroticism and positive emotions. Personality and Individual Differences 47, 1 (2009), 69--72.

[14]

Perugini, M. and Blas, L. Di. Analyzing personality related adjectives from an etic-emic perspective: The Big Five Marker Scales (BFMS) and the Italian AB5C taxonomy. Big Five Assessment, (2002), 281--304.

[15]

Sinha, R. Multivariate Response Patterning of Fear and Anger. Cognition & Emotion 10, 2 (1996), 173--198.

[16]

Soleymani, M., Lichtenauer, J., Pun, T., and Pantic, M. A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing 3, 1 (2012), 42--55.

Digital Library

[17]

Srivastava, R., Feng, J., Roy, S., Sim, T., and Yan, S. Don't Ask Me What I'm Like, Just Watch and Listen. Proceedings of the 20th ACM international conference on Multimedia. ACM, 2012. (2012), 329--338.

Digital Library

[18]

Tok, S., Koyuncu, M., Dural, S., and Catikkas, F. Evaluation of International Affective Picture System (IAPS) ratings in an athlete population and its relations to personality. Personality and Individual Differences 49, 5 (2010), 461--466.

[19]

Zeng, Z., Pantic, M., Roisman, G. I., and Huang, T. S. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on pattern analysis and machine intelligence 31, 1 (2009), 39--58.

Digital Library

Cited By

Riaz MGravina R(2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JSEN.2024.3429357
Wang YLi DFunakoshi KOkumura M(2023)EMP: Emotion-guided Multi-modal Fusion and Contrastive Learning for Personality Traits RecognitionProceedings of the 2023 ACM International Conference on Multimedia Retrieval10.1145/3591106.3592243(243-252)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3591106.3592243
Vyas PDesai UYamakawa KMaclean K(2023)A Descriptive Analysis of a Formative Decade of Research in Affective Haptic System DesignProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580735(1-23)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580735
Show More Cited By

Index Terms

The Secret Language of Our Body: Affect and Personality Recognition Using Physiological Signals
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Quantifying the Links Between Personality Sub-traits and the Basic Emotions
Computational Science and Its Applications – ICCSA 2020
Abstract
This article describes an exploratory study that aimed to analyse the relationship between personality traits and emotions. In particular, it investigates to what extent the sub-traits of the Five Factor Model has an empirically quantifiable ...
Pupil size variation as an indication of affective processing
Application of affective computing in humanComputer interaction

The present objective was to investigate pupil size variation during and after auditory emotional stimulation. Thirty subjects' (15 females and 15 males) pupil responses were measured while listening to 10 negative and 10 positive highly arousing sounds ...
Expressing emotions as emoticons for online intelligent agents
HCI '16: Proceedings of the 30th International BCS Human Computer Interaction Conference: Fusion!

Without emotional annotation, online communication can be ambiguous and lead to misunderstandings. This paper addresses the questions of which emotions are commonly expressed online, how these emotions can be encapsulated in emoticons, and how people ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

November 2014

558 pages

ISBN:9781450328852

DOI:10.1145/2663204

General Chairs:
Albert Ali Salah
Boğaziçi University, Turkey
,
Jeffrey Cohn
University of Pittsburgh, USA
,
Björn Schuller
University of Passau, Germany and Imperial College London, UK
,
Program Chairs:
Oya Aran
Idiap Research Institute, Switzerland
,
Louis-Philippe Morency
University of Southern California, USA
,
Philip R. Cohen
Adapx, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Extended-abstract

Conference

ICMI '14

Sponsor:

SIGCHI

ICMI '14: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 12 - 16, 2014

Istanbul, Turkey

Acceptance Rates

ICMI '14 Paper Acceptance Rate 51 of 127 submissions, 40%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
440
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)7

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Riaz MGravina R(2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JSEN.2024.3429357
Wang YLi DFunakoshi KOkumura M(2023)EMP: Emotion-guided Multi-modal Fusion and Contrastive Learning for Personality Traits RecognitionProceedings of the 2023 ACM International Conference on Multimedia Retrieval10.1145/3591106.3592243(243-252)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3591106.3592243
Vyas PDesai UYamakawa KMaclean K(2023)A Descriptive Analysis of a Formative Decade of Research in Affective Haptic System DesignProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580735(1-23)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580735
Tseng CLin HHuang ALin J(2023)Personalized programming education: Using machine learning to boost learning performance based on students’ personality traitsCogent Education10.1080/2331186X.2023.224563710:2Online publication date: 9-Aug-2023
https://doi.org/10.1080/2331186X.2023.2245637
Tseng CLin HHuang AChen YLin J(2023)MindMe: an AI-Powered personality assessment toolMultimedia Tools and Applications10.1007/s11042-023-16803-x83:12(35943-35955)Online publication date: 28-Sep-2023
https://doi.org/10.1007/s11042-023-16803-x
Singh JArya R(2023)Examining the relationship of personality traits with online teaching using emotive responses and physiological signalsEducation and Information Technologies10.1007/s10639-023-11619-628:9(11193-11219)Online publication date: 14-Feb-2023
https://doi.org/10.1007/s10639-023-11619-6
Pan YXu J(2023)Personality Traits Inference in the Hybrid Foraging Search TaskDesign, User Experience, and Usability10.1007/978-3-031-35702-2_19(258-269)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35702-2_19
Cabada RLópez HEscalante HCabada RLópez HEscalante H(2023)Personality Recognition ModelsMultimodal Affective Computing10.1007/978-3-031-32542-7_14(167-171)Online publication date: 21-Apr-2023
https://doi.org/10.1007/978-3-031-32542-7_14
Zhao XTang ZZhang S(2022)Deep Personality Trait Recognition: A SurveyFrontiers in Psychology10.3389/fpsyg.2022.83961913Online publication date: 6-May-2022
https://doi.org/10.3389/fpsyg.2022.839619
Wei YZhong ZGu JOshana R(2022)Human emotion based real-time memory and computation management on resource-limited edge devicesProceedings of the 59th ACM/IEEE Design Automation Conference10.1145/3489517.3530490(487-492)Online publication date: 10-Jul-2022
https://dl.acm.org/doi/10.1145/3489517.3530490
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents