research-article

Wearable Emotion Recognition System based on GSR and PPG Signals

Authors:

Goran Udovičić,

Marjan SikoraAuthors Info & Claims

MMHealth '17: Proceedings of the 2nd International Workshop on Multimedia for Personal Health and Health Care

Pages 53 - 59

https://doi.org/10.1145/3132635.3132641

Published: 23 October 2017 Publication History

Abstract

In recent years, many methods and systems for automated recognition of human emotional states were proposed. Most of them are trying to recognize emotions based on physiological signals such as galvanic skin response (GSR), electrocardiogram (ECG), electroencephalogram (EEG), electromyogram (EMG), photoplethysmogram (PPG), respiration, skin temperature etc. Measuring all these signals is quite impractical for real-life use and in this research, we decided to acquire and analyse only GSR and PPG signals because of its suitability for implementation on a simple wearable device that can collect signals from a person without compromising comfort and privacy. For this purpose, we used the lightweight, small and compact Shimmer3 sensor. We developed complete application with database storage to elicit participant»s emotions using pictures from the Geneva affective picture database (GAPED) database. In the post-processing process, we used typical statistical parameters and power spectral density (PSD) as features and support vector machine (SVM) and k-nearest neighbours (KNN) as classifiers. We built single-user and multi-user emotion classification models to compare the results. As expected, we got better average accuracies on a single-user model than on the multi-user model. Our results also show that a single-user based emotion detection model could potentially be used in real-life scenario considering environments conditions.

References

[1]

P. Ekman and W. Friesen. 1982. Measuring facial movement with the facial action coding system. In Emotion in the human face (2nd ed.), New York: Cambridge University Press

[2]

Rosalind W. Picard. 1997. Affective computing. MIT Press.

Digital Library

[3]

R. W. Picard, J. Healey. 1997 Affective wearables. vol. 1, Issue: 4, pp 231--240.

[4]

M. Monajati, S.H. Abbasi, F. Shabaninia. 2012. Emotions States Recognition Based on Physiological Parameters by Employing of Fuzzy-Adaptive Resonance Theory. International Journal of Intelligence Science, 2012, 2, 166--175.

[5]

C. Maaoui, A. Pruski. 2010. Emotion Recognition through Physiological Signals for Human-Machine Communication. Cutting Edge Robotics 2010, Vedran Kordic (Ed.), ISBN: 978--953--307-062--9, InTech

[6]

A.M. Khan, M. Lawo. 2016. Wearable Recognition System for Emotional States Using Physiological Devices. eTELEMED 2016 : The Eighth International Conference on eHealth, Telemedicine, and Social Medicine, ISBN: 978--1--61208--470--1 131

[7]

P. Das, A. Khasnobish, D.N. Tibarewala. 2016. Emotion Recognition employing ECG and GSR Signals as Markers of ANS. Conference on Advances in Signal Processing (CASP)

[8]

G. Wu, G. Liu, M. Hao. 2010. The analysis of emotion recognition from GSR based on PSO. In Intelligence Information Processing and Trusted Computing (IPTC), 28--29 Oct. 2010

Digital Library

[9]

K.H.Kim, S.W.Bang, S.R. Kim. 2014. Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput., 2004, 42, 419--427

[10]

N. Amour, A. Hersi, N. Alajlan. 2004. Implementation of a Mobile Health System for Monitoring ECG signals. BioMedCom 2014 Conference, Harvard University, December 14--16

[11]

E. Fortune, M. Tierney, C. Ni Scanaill. 2011. Activity Level Classification Algorithm Using SHIMMERTM Wearable Sensors for Individuals with Rheumatoid Arthritis. 33rd Annual International Conference of the IEEE EMBS Boston, Massachusetts USA

[12]

R. Richer, P. Blank, D. Schuldhaus. 2014. Real-time ECG and EMG Analysis for Biking using Android-based Mobile Devices. 11th International Conference on Wearable and Implantable Body Sensor Networks

Digital Library

[13]

A. Nakasone, H. Prendinger, M. Ishizuka. 2005. Emotion recognition from electromyography and skin conductance. In Proceeding. of the 5th International Workshop on Biosignal Interpretation (pp. 219--222)

[14]

M. Soleymani, J. Lichtenauer,T. Pun. 2012. A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transaction on Affective Computing, Vol. 3, No. 1

Digital Library

[15]

R. Subramanian, J. Wache, M.K. Abadi. 2016. ASCERTAIN: Emotion and Personality Recognition using Commercial Sensors. J. IEEE Transaction on affective computing, Vol. 3, No. 1

[16]

M. van Dooren, J.J.G. de Vries, J. H. Janssen. 2012. Emotional sweating across the body: Comparing 16 different skin conductance measurement locations. Physiology & Behavior 106 (2012) 298--304,

[17]

M. Elgendi. 2012. On the Analysis of Fingertip Photoplethysmogram Signals. In Intelligence Information Processing and Trusted Computing (IPTC), J. Curr Cardiol Rev. 2012 Feb; 8(1): 14--25

[18]

A. Burns, E.P. Doheny, B.R. Greene. 2010. SHIMMER™: An extensible platform for physiological signal capture. In Engineering in Medicine and Biology Society (EMBC), Annual International Conference of the IEEE.

[19]

E. S. Dan-Glauser and K. R. Scherer. 2011. The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance. Behavior Research Methods, vol. 43, no. 2, pp. 468--477

[20]

M.M. Bradley and P.J. Lang. 1994. Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. J. Behav. Ther. Exp. Psychiatry, Vol. 25, No. I. pp. 49--59

[21]

J. Wagner, J. Kim, E. André. 2005. From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification. In IEEE International Conference on Multimedia & Expo (ICME 2005)

[22]

J. Cheng, G.Y. Liu. 2015. Affective detection based on an imbalanced fuzzy support vector machine. Biomedical Signal Processing and Control, pp.118--126

[23]

C.Y. Chang, C.W. Chang, Y.M. Lin. 2012. Application of support vector machine for emotion classification. In Genetic and Evolutionary Computing (ICGEC), 2012 Sixth International Conference on, IEEE, pp.249--252

Digital Library

[24]

R. Levenson. 1988. Emotion and the autonomic nervous system: a prospectus for research on autonomic specificity. In H. Wagner, editor, Social Psychophysiology and Emotion: Theory and Clinical Applications, pages 17--42. John Wiley & Sons

Cited By

Gebhardt CBrombach ALuong THilliges OHolz C(2024)Detecting Users' Emotional States during Passive Social Media UseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596068:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659606
Su YHuang SLiu HChen YDu YWang YRen YChen Y(2024)PPG-Hear: A Practical Eavesdropping Attack with Photoplethysmography SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596038:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659603
Luo YYu JLiang MWan YZhu KSantosa S(2024)Emotion Embodied: Unveiling the Expressive Potential of Single-Hand GesturesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642255(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642255
Show More Cited By

Index Terms

Wearable Emotion Recognition System based on GSR and PPG Signals
1. Applied computing
  1. Life and medical sciences
    1. Bioinformatics

Recommendations

Emotion detection and recognition using HRV features derived from photoplethysmogram signals
ERM4CT '16: Proceedings of the 2nd workshop on Emotion Representations and Modelling for Companion Systems

Detection of true human emotions has attracted a lot of interest in the recent years. The applications range from e-retail to health-care for developing effective companion systems with reliable emotion recognition. This paper proposes heart rate ...
Emotion Recognition Using Physiological Signals
MIDI '15: Proceedings of the Mulitimedia, Interaction, Design and Innnovation

In this paper the problem of emotion recognition using physiological signals is presented. Firstly the problems with acquisition of physiological signals related to specific human emotions are described. It is not a trivial problem to elicit real ...
Affect recognition based on physiological changes during the watching of music videos
Special Issue on Affective Interaction in Natural Environments

Assessing emotional states of users evoked during their multimedia consumption has received a great deal of attention with recent advances in multimedia content distribution technologies and increasing interest in personalized content delivery. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMHealth '17: Proceedings of the 2nd International Workshop on Multimedia for Personal Health and Health Care

October 2017

104 pages

ISBN:9781450355049

DOI:10.1145/3132635

Program Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Touradj Ebrahimi
EPFL, Switzerland
,
Cathal Gurrin
Dublin City University, Ireland
,
Ramesh Jain
UC Irvine, USA
,
Laleh Jalali
Hitachi America Ltd. USA
,
Jochen Meyer
OFFIS-Institute for Information Technology, Germany
,
Noel O'Connor
Dublin City University, Ireland

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Hrvatska Zaklada za Znanost

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23, 2017

California, Mountain View, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
1,950
Total Downloads

Downloads (Last 12 months)418
Downloads (Last 6 weeks)44

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gebhardt CBrombach ALuong THilliges OHolz C(2024)Detecting Users' Emotional States during Passive Social Media UseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596068:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659606
Su YHuang SLiu HChen YDu YWang YRen YChen Y(2024)PPG-Hear: A Practical Eavesdropping Attack with Photoplethysmography SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596038:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659603
Luo YYu JLiang MWan YZhu KSantosa S(2024)Emotion Embodied: Unveiling the Expressive Potential of Single-Hand GesturesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642255(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642255
Sun JPortilla JOtero A(2024)A Deep Learning Approach for Fear Recognition on the Edge Based on Two-Dimensional Feature MapsIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.339237328:7(3973-3984)Online publication date: Jul-2024
https://doi.org/10.1109/JBHI.2024.3392373
Boljanić TŠtrbac MKostić MMalešević J(2024)Overview of the Methods and Applications of Electrodermal Activity Assessment and Its Relation with Electrotactile Feedback and Potential Use in Automated Calibration2024 11th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN)10.1109/IcETRAN62308.2024.10645183(1-6)Online publication date: 3-Jun-2024
https://doi.org/10.1109/IcETRAN62308.2024.10645183
Apurupa MSngh H(2024)EmotiGuard: Real-Time Human Emotion Recognition System for Adult Monitoring2024 International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT)10.1109/IConSCEPT61884.2024.10627881(1-6)Online publication date: 4-Jul-2024
https://doi.org/10.1109/IConSCEPT61884.2024.10627881
Zhang RFang RHosseini EFang CMiao NHomayoun H(2024)Advanced Energy-Efficient System for Precision Electrodermal Activity Monitoring in Stress Detection2024 IEEE 20th International Conference on Body Sensor Networks (BSN)10.1109/BSN63547.2024.10780551(1-4)Online publication date: 15-Oct-2024
https://doi.org/10.1109/BSN63547.2024.10780551
Pancholi SEverett TDuerstock B(2024)Advancing spinal cord injury care through non-invasive autonomic dysreflexia detection with AIScientific Reports10.1038/s41598-024-53718-514:1Online publication date: 10-Feb-2024
https://doi.org/10.1038/s41598-024-53718-5
Choi GShin JLee JLee JHeo IYoon HLim WJeong JKim SHwang H(2024)EEG Dataset for the Recognition of Different Emotions Induced in Voice-User InteractionScientific Data10.1038/s41597-024-03887-911:1Online publication date: 3-Oct-2024
https://doi.org/10.1038/s41597-024-03887-9
Yang PLiu NLiu XShu YJi WRen ZSheng JYu MYi RZhang DLiu Y(2024)A Multimodal Dataset for Mixed Emotion RecognitionScientific Data10.1038/s41597-024-03676-411:1Online publication date: 5-Aug-2024
https://doi.org/10.1038/s41597-024-03676-4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents