article

Real-time EEG-based emotion monitoring using stable features

Authors:

Yisi LiuAuthors Info & Claims

The Visual Computer: International Journal of Computer Graphics, Volume 32, Issue 3

Pages 347 - 358

https://doi.org/10.1007/s00371-015-1183-y

Published: 01 March 2016 Publication History

Abstract

In human---computer interaction (HCI), electroencephalogram (EEG) signals can be added as an additional input to computer. An integration of real-time EEG-based human emotion recognition algorithms in human---computer interfaces can make the users experience more complete, more engaging, less emotionally stressful or more stressful depending on the target of the applications. Currently, the most accurate EEG-based emotion recognition algorithms are subject-dependent, and a training session is needed for the user each time right before running the application. In this paper, we propose a novel real-time subject-dependent algorithm with the most stable features that gives a better accuracy than other available algorithms when it is crucial to have only one training session for the user and no re-training is allowed subsequently. The proposed algorithm is tested on an affective EEG database that contains five subjects. For each subject, four emotions (pleasant, happy, frightened and angry) are induced, and the affective EEG is recorded for two sessions per day in eight consecutive days. Testing results show that the novel algorithm can be used in real-time emotion recognition applications without re-training with the adequate accuracy. The proposed algorithm is integrated with real-time applications "Emotional Avatar" and "Twin Girls" to monitor the users emotions in real time.

References

[1]

Allen, J.J., Urry, H.L., Hitt, S.K., Coan, J.A.: The stability of resting frontal electroencephalographic asymmetry in depression. Psychophysiology 41(2), 269---280 (2004)

[2]

Bouton, M.E.: Learning and Behavior: A Contemporary Synthesis. Sinauer Associates, Sunderland (2007)

[3]

Bradley, M.M., Lang, P.J.: The international affective digitized sounds (; iads-2): affective ratings of sounds and instruction manual. Technical report B-3, University of Florida, Gainesville (2007)

[4]

Brown, L., Grundlehner, B., Penders, J.: Towards wireless emotional valence detection from EEG. In: Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp. 2188---2191. IEEE (2011)

[5]

Chanel, G., Kierkels, J.J., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. Int. J. Hum.---Comput. Stud. 67(8), 607---627 (2009)

Digital Library

[6]

Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

Digital Library

[7]

Emotiv. http://www.emotiv.com. Accessed 26 Feb 2015

[8]

Frantzidis, C., Bratsas, C., Papadelis, C.L., Konstantinidis, E., Pappas, C., Bamidis, P.D., et al.: Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli. Inf. Technol. Biomed. IEEE Trans. 14(3), 589---597 (2010)

Digital Library

[9]

Gasser, T., Bächer, P., Steinberg, H.: Test-retest reliability of spectral parameters of the EEG. Electroencephalogr. Clin. Neurophysiol. 60(4), 312---319 (1985)

[10]

Gasser, T., Jennen-Steinmetz, C., Verleger, R.: EEG coherence at rest and during a visual task in two groups of children. Electroencephalogr. Clin. Neurophysiol. 67(2), 151---158 (1987)

[11]

Gudmundsson, S., Runarsson, T.P., Sigurdsson, S., Eiriksdottir, G., Johnsen, K.: Reliability of quantitative EEG features. Clin. Neurophysiol. 118(10), 2162---2171 (2007)

[12]

Haptek. http://www.haptek.com. Accessed 28 Feb 2015

[13]

Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Phys. D: Nonlinear Phenom. 31(2), 277---283 (1988)

Digital Library

[14]

Hosseini, S.A., Khalilzadeh, M.A., Naghibi-Sistani, M.B., Niazmand, V.: Higher order spectra analysis of EEG signals in emotional stress states. In: 2010 Second International Conference on Information Technology and Computer Science (ITCS), pp. 60---63. IEEE (2010)

Digital Library

[15]

Ishino, K., Hagiwara, M.: A feeling estimation system using a simple electroencephalograph. In: IEEE International Conference on Systems, Man and Cybernetics, 2003. vol. 5, pp. 4204---4209. IEEE (2003)

[16]

Kedem, B., Yakowitz, S.: Time Series Analysis by Higher Order Crossings. IEEE Press, Piscataway (1994)

[17]

Koelstra, S., Mühl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: Deap: a database for emotion analysis; using physiological signals. Affect. Comput. IEEE Trans. 3(1), 18---31 (2012)

Digital Library

[18]

Kondacs, A., Szabó, M.: Long-term intra-individual variability of the background EEG in normals. Clin. Neurophysiol. 110(10), 1708---1716 (1999)

[19]

Lan, Z., Sourina, O., Wang, L., Liu, Y.: Stability of features in real-time EEG-based emotion recognition algorithm. In: 2014 International Conference on Cyberworlds (CW), pp. 137---144. IEEE (2014)

Digital Library

[20]

Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (iaps): affective ratings of pictures and instruction manual. Technical report A-8 (2008)

[21]

Li, M., Lu, B.L.: Emotion classification based on gamma-band EEG. In: Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, pp. 1223---1226. IEEE (2009)

[22]

Lin, Y.P., Wang, C.H., Jung, T.P., Wu, T.L., Jeng, S.K., Duann, J.R., Chen, J.H.: EEG-based emotion recognition in music listening. Biomed. Eng. IEEE Trans. 57(7), 1798---1806 (2010)

[23]

Lin, Y.P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H.: EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2009. ICASSP 2009, pp. 489---492. IEEE (2009)

Digital Library

[24]

Liu, Y., Sourina, O.: EEG databases for emotion recognition. In: 2013 International Conference on Cyberworlds (CW), pp. 302---309. IEEE (2013)

Digital Library

[25]

Liu, Y., Sourina, O.: Real-time fractal-based valence level recognition from EEG. In: Transactions on Computational Science XVIII, pp. 101---120. Springer (2013)

[26]

Liu, Y., Sourina, O.: Real-time subject-dependent EEG-based emotion recognition algorithm. In: Transactions on Computational Science XXIII, pp. 199---223. Springer (2014)

[27]

Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based emotion recognition and its applications. In: Transactions on computational science XII, pp. 256---277. Springer (2011)

[28]

McGraw, K.O., Wong, S.P.: Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1(1), 30 (1996)

[29]

Murugappan, M., Nagarajan, R., Yaacob, S.: Combining spatial filtering and wavelet transform for classifying human emotions using EEG signals. J. Med. Biol. Eng. 31(1), 45---51 (2011)

[30]

Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. Inf. Technol. Biomed. IEEE Trans. 14(2), 186---197 (2010)

Digital Library

[31]

Petrantonakis, P.C., Hadjileontiadis, L.J.: A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. Inf. Technol. Biomed. IEEE Trans. 15(5), 737---746 (2011)

Digital Library

[32]

Petrantonakis, P.C., Hadjileontiadis, L.J.: Adaptive emotional information retrieval from EEG signals in the time-frequency domain. Signal Process. IEEE Trans. 60(5), 2604---2616 (2012)

Digital Library

[33]

Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. Pattern Anal. Mach. Intell. IEEE Trans. 23(10), 1175---1191 (2001)

Digital Library

[34]

Salinsky, M., Oken, B., Morehead, L.: Test-retest reliability in EEG frequency analysis. Electroencephalogr. Clin. Neurophysiol. 79(5), 382---392 (1991)

[35]

Sanei, S., Chambers, J.A.: EEG Signal Processing. Wiley, New York (2013)

[36]

Schaaff, K.: EEG-Based Emotion Recognition. Universitat Karlsruhe (TH), Karlsruhe (2008)

[37]

Schaaff, K., Schultz, T.: Towards an EEG-based emotion recognizer for humanoid robots. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, 2009. RO-MAN 2009, pp. 792---796. IEEE (2009)

[38]

Sohaib, A.T., Qureshi, S., Hagelbäck, J., Hilborn, O., Jeră¿ić, P.: Evaluating classifiers for emotion recognition using EEG. In: Foundations of Augmented Cognition, pp. 492---501. Springer (2013)

[39]

Sourina, O., Liu, Y., Nguyen, M.K.: Real-time EEG-based emotion recognition for music therapy. J. Multimodal User Interf. 5(1---2), 27---35 (2012)

[40]

Tomarken, A.J., Davidson, R.J., Wheeler, R.E., Kinney, L.: Psychometric properties of resting anterior EEG asymmetry: temporal stability and internal consistency. Psychophysiology 29(5), 576---592 (1992)

[41]

Wang, X.W., Nie, D., Lu, B.L.: EEG-based emotion recognition using frequency domain features and support vector machines. In: Neural Information Processing, pp. 734---743. Springer (2011)

Digital Library

[42]

Williams, C.E., Stevens, K.N.: Emotions and speech: some acoustical correlates. J. Acoust. Soc. Am. 52(4B), 1238---1250 (1972)

Cited By

Garg DVerma GSingh A(2024)EEG-based emotion recognition using MobileNet Recurrent Neural Network with time-frequency featuresApplied Soft Computing10.1016/j.asoc.2024.111338154:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111338
Bardak FSeyman MTemurtaş F(2024)Adaptive neuro-fuzzy based hybrid classification model for emotion recognition from EEG signalsNeural Computing and Applications10.1007/s00521-024-09573-636:16(9189-9202)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00521-024-09573-6
Wang YYu HGao WXia YNduka C(2023)MGEED: A Multimodal Genuine Emotion and Expression Detection DatabaseIEEE Transactions on Affective Computing10.1109/TAFFC.2023.328635115:2(606-619)Online publication date: 15-Jun-2023
https://dl.acm.org/doi/10.1109/TAFFC.2023.3286351
Show More Cited By

Recommendations

Stability of Features in Real-Time EEG-based Emotion Recognition Algorithm
CW '14: Proceedings of the 2014 International Conference on Cyberworlds

Stability of algorithms is very important for electroencephalogram (EEG) based applications. Stable features should exhibit consistency among repeated measurements of the same subject. Previously, power features were reported to be one of the most ...
EEG-based Valence Level Recognition for Real-Time Applications
CW '12: Proceedings of the 2012 International Conference on Cyberworlds

Emotions are important in human-computer interaction. Emotions could be classified based on 3-dimensional Valence-Arousal-Dominance model which allows defining any number of emotions even without discrete emotion labels. In this paper, we proposed a ...
Real-time EEG-based emotion recognition and its applications
Transactions on computational science XII

Since emotions play an important role in the daily life of human beings, the need and importance of automatic emotion recognition has grown with increasing role of human computer interface applications. Emotion recognition could be done from the text, ...

Comments

Information & Contributors

Information

Published In

cover image The Visual Computer: International Journal of Computer Graphics

The Visual Computer: International Journal of Computer Graphics Volume 32, Issue 3

March 2016

137 pages

ISSN:0178-2789

Issue’s Table of Contents

Copyright © Copyright © 2016 Springer-Verlag Berlin Heidelberg.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 March 2016

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Garg DVerma GSingh A(2024)EEG-based emotion recognition using MobileNet Recurrent Neural Network with time-frequency featuresApplied Soft Computing10.1016/j.asoc.2024.111338154:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111338
Bardak FSeyman MTemurtaş F(2024)Adaptive neuro-fuzzy based hybrid classification model for emotion recognition from EEG signalsNeural Computing and Applications10.1007/s00521-024-09573-636:16(9189-9202)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00521-024-09573-6
Wang YYu HGao WXia YNduka C(2023)MGEED: A Multimodal Genuine Emotion and Expression Detection DatabaseIEEE Transactions on Affective Computing10.1109/TAFFC.2023.328635115:2(606-619)Online publication date: 15-Jun-2023
https://dl.acm.org/doi/10.1109/TAFFC.2023.3286351
Garima Goel NRathee N(2023)Modified multidimensional scaling on EEG signals for emotion classificationMultimedia Tools and Applications10.1007/s11042-023-14671-z82:18(28547-28568)Online publication date: 23-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14671-z
Kamble KSengupta J(2023)A comprehensive survey on emotion recognition based on electroencephalograph (EEG) signalsMultimedia Tools and Applications10.1007/s11042-023-14489-982:18(27269-27304)Online publication date: 9-Feb-2023
https://dl.acm.org/doi/10.1007/s11042-023-14489-9
Li XZhang YTiwari PSong DHu BYang MZhao ZKumar NMarttinen P(2022)EEG Based Emotion Recognition: A Tutorial and ReviewACM Computing Surveys10.1145/352449955:4(1-57)Online publication date: 21-Nov-2022
https://dl.acm.org/doi/10.1145/3524499
Ngai WXie HZou DChou K(2022)Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sourcesInformation Fusion10.1016/j.inffus.2021.07.00777:C(107-117)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1016/j.inffus.2021.07.007
Maithri MRaghavendra UGudigar ASamanth JPrabal Datta Barua Murugappan MChakole YAcharya U(2022)Automated emotion recognitionComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2022.106646215:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.cmpb.2022.106646
Wang JShao HYao YLiu JSun HMa S(2022)Electroencephalograph-based emotion recognition using convolutional neural network without manual feature extractionApplied Soft Computing10.1016/j.asoc.2022.109534128:COnline publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.109534
Santhiya PChitrakala S(2022)PTCERE: personality-trait mapping using cognitive-based emotion recognition from electroencephalogram signalsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02502-539:7(2953-2967)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.1007/s00371-022-02502-5
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents