Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

SpringerLink
Log in

Emotion recognition based on phase-locking value brain functional network and topological data analysis

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Traditional threshold-based methods in brain functional network analysis have some drawbacks. First, the process of determining thresholds is often based on trial and error, lacking standardization, and exhibiting strong subjectivity. Second, this subjectivity may lead to the loss of emotion-related information, limiting a comprehensive understanding and accurate identification of underlying neural processes. To overcome these problems, the persistent homology (PH) theory based on topological data analysis was introduced in this study, and a PH-based framework for emotion recognition in functional brain networks was proposed. Firstly, the EEG signals were divided into five frequency bands (\(\delta\), \(\theta\), \(\alpha\), \(\beta\), and \(\gamma\)) and segmented into multiple time series using non-overlapping sliding windows. Secondly, considering the coupling relationship between brain regions in different emotional states, the degree of phase synchronization between channels was calculated using the phase-locking value (PLV), and the PLV-based brain functional network was constructed. The PLV brain functional network was then analyzed with PH to extract multiple persistent topological features and combine them into richer feature vectors. These persistent topological features include persistence landscapes, Betti curves, persistent entropy, amplitudes, and non-diagonal points. Finally, these persistent feature vectors are used as inputs to a classifier and used for emotion recognition using machine learning algorithms and majority voting methods. Experimental analyses were conducted on the DEAP dataset to evaluate the proposed model. Further validation of the model was implemented on the DREAMER dataset and SEED dataset. The results show that the model achieves good results in EEG emotion recognition. The average accuracy reached 89.94, 87.61, and 83.24%, respectively. In this study, we extracted potential features of EEG data by applying PH to the exploration of functional brain networks, which avoided the reliance on manually determined thresholds and enabled a more comprehensive and accurate method of emotion recognition. Compared with traditional functional brain network methods, our method retains more original information related to functional brain networks in a stable and threshold-free manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

All code and data are available.

References

  1. Kang Q, Gao Q, Song Y, Tian Z, Yang Y, Mao Z, Dong E (2021) Emotion recognition from deaf eeg signals using stacking ensemble learning frame-work based on a novel brain network. IEEE Sens J, 08:1–1

  2. Mauss IB, Robinson MD (2009) Measures of emotion: a review. Cognit Emotion 23(2):209–237

    Google Scholar 

  3. Mao Y, Song L, Zeng X (2021) Emotion classification from short-term eeg signals in deep learning. In: 2021 3rd International symposium on smart and healthy cities (ISHC), pp 169–173. IEEE

  4. Park C, Ryu J, Sohn J, Cho H (2007) An emotion expression system for the emotional robot. In: 2007 IEEE international symposium on consumer electronics, pp 1–6

  5. Zheng W-L, Zhu J-Y, Lu B-L (2017) Identifying stable patterns over time for emotion recognition from eeg. IEEE Trans Affect Comput 10(3):417–429

    Google Scholar 

  6. Chen T, Ju S, Yuan X, Elhoseny M, Ren F, Fan M, Chen Z (2018) Emotion recognition using empirical mode decomposition and approximation entropy. Comput Electrical Eng 72:383–392

    Google Scholar 

  7. Zheng W-L, Guo H-T, Lu B-L (2015) Revealing critical channels and frequency bands for emotion recognition from eeg with deep belief network. In: 2015 7th International IEEE/EMBS conference on neural engineering (NER), pp 154–157. IEEE

  8. Yin Y, Zheng X, Hu B, Zhang Y, Cui X (2021) Eeg emotion recognition using fusion model of graph convolutional neural networks and lstm. Appl Soft Comput 100:106954

    Google Scholar 

  9. Ahmadlou M, Adeli H (2011) Functional community analysis of brain: a new approach for eeg-based investigation of the brain pathology. Neuroimage 58(2):401–408

    Google Scholar 

  10. Wang ZM, Zhou R, He Y, Guo XM (2020) Functional integration and separation of brain network based on phase locking value during emotion processing. IEEE Trans Cognit Develop Syst, 99:1

  11. Xefteris V-R, Tsanousa A, Georgakopoulou N, Diplaris S, Vrochidis S, Kompatsiaris I (2022) Graph theoretical analysis of eeg functional connectivity patterns and fusion with physiological signals for emotion recognition. Sensors 22(21):8198

    Google Scholar 

  12. Liu Y, Fu G (2021) Emotion recognition by deeply learned multi-channel textual and eeg features. Future Generation Comput Syst 119:1–6

    Google Scholar 

  13. Adamovich T, Zakharov I, Tabueva A, Malykh S (2022) The thresholding problem and variability in the eeg graph network parameters. Sci Rep 12(1):18659

    Google Scholar 

  14. Katmah R, Al-Shargie F, Tariq U, Babiloni F, Al-Mughairbi F, Al-Nashash H (2023) Mental stress management using fnirs directed connectivity and audio stimulation. IEEE Trans Neural Syst Rehabil Eng 31:1086–1096

    Google Scholar 

  15. Al-Shargie F, Katmah R, Tariq U, Babiloni F, Al-Mughairbi F, Al-Nashash H (2022) Stress management using fnirs and binaural beats stimulation. Biomed Opt Exp 13(6):3552–3575

    Google Scholar 

  16. Xiaoqi X, Drougard N, Roy RN (2021) Topological data analysis as a new tool for eeg processing. Front Neurosci 15:761703

    Google Scholar 

  17. Wagner H, Dłotko P, Mrozek M (2012) Computational topology in text mining. In: Computational topology in image context: 4th international workshop, CTIC 2012, Bertinoro, Italy, May 28–30, 2012. Proceedings, pp 68–78. Springer

  18. Seversky LM, Davis S, Berger M (2016) On time-series topological data analysis: new data and opportunities. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 59–67

  19. Sizemore AE, Phillips-Cremins JE, Ghrist R, Bassett DS (2019) The importance of the whole: topological data analysis for the network neuroscientist. Network Neurosci 3(3):656–673

    Google Scholar 

  20. Khalid A, Kim BS, Chung MK, Ye JC, Jeon D (2014) Tracing the evolution of multi-scale functional networks in a mouse model of depression using persistent brain network homology. NeuroImage 101:351–363

    Google Scholar 

  21. Guo G, Zhao Y, Liu C, Fu Y, Xi X, Jin L, Shi D, Wang L, Duan Y, Huang J et al (2022) Method for persistent topological features extraction of schizophrenia patients’ electroencephalography signal based on persistent homology. Front Comput Neurosci 16:1024205

    Google Scholar 

  22. Wang Y, Ombao H, Chung MK (2018) Topological data analysis of single-trial electroencephalographic signals. Ann Appl Stat 12(3):1506

    MathSciNet  Google Scholar 

  23. Yan Y, Wu X, Li C, He Y, Zhang Z, Li H, Li A, Wang L (2023) Topological eeg nonlinear dynamics analysis for emotion recognition. IEEE Trans Cognit Develop Syst 15(2):625–638

    Google Scholar 

  24. Chazal F, Michel B (2021) An introduction to topological data analysis: fundamental and practical aspects for data scientists. Front Artif Intell 4:108

    Google Scholar 

  25. Chen C, Li Z, Wan F, Leicai X, Bezerianos A, Wang H (2022) Fusing frequency-domain features and brain connectivity features for cross-subject emotion recognition. IEEE Trans Instrument Measure 71:1–15

    Google Scholar 

  26. Hjorth B (1970) Eeg analysis based on time domain properties. Electroencephalogr Clin Neurophysiol 29(3):306–310

    Google Scholar 

  27. Liu Y, Sourina O (2013) Real-time fractal-based valence level recognition from eeg. In: Transactions on computational science XVIII: special issue on cyberworlds, pp 101–120. Springer

  28. Petrantonakis PC, Hadjileontiadis LJ (2009) Emotion recognition from eeg using higher order crossings. IEEE Trans Inf Technol Biomed 14(2):186–197

    Google Scholar 

  29. Nie D, Wang X-W, Shi L-C, Lu B-L (2011) Eeg-based emotion recognition during watching movies. In: 2011 5th International IEEE/EMBS conference on neural engineering, pp 667–670. IEEE

  30. Shi L-C, Jiao Y-Y, Lu B-L (2013) Differential entropy feature for eeg-based vigilance estimation. In: 2013 35th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 6627–6630. IEEE

  31. Wang X-W, Nie D, Lu B-L (2014) Emotional state classification from eeg data using machine learning approach. Neurocomputing 129:94–106

    Google Scholar 

  32. Pei Z, Wang H, Bezerianos A, Li J (2020) Eeg-based multiclass workload identification using feature fusion and selection. IEEE Trans Instrument Measure 70:1–8

    Google Scholar 

  33. Wang H, Liu X, Li J, XuT, Bezerianos A, Sun Y, Wan F (2020) Driving fatigue recognition with functional connectivity based on phase synchronization. IEEE Trans Cognit Develop Syst 13(3):668–678

  34. Wang Z, Tong Y, Heng X (2019) Phase-locking value based graph convolutional neural networks for emotion recognition. IEEE Access 7:93711–93722

    Google Scholar 

  35. Wang Z-M, Chen Z-Y, Zhang J (2023) Eeg emotion recognition based on plv-rich-club dynamic brain function network. Appl Intell 53(14):17327–17345

    Google Scholar 

  36. Wang Z-M, Zhang J-W, He Y, Zhang J (2022) Eeg emotion recognition using multichannel weighted multiscale permutation entropy. Appl Intell 52(10):12064–12076

    Google Scholar 

  37. Li P, Liu H, Si Y, Li C, Li F, Zhu X, Huang X, Zeng Y, Yao D, Zhang Y et al (2019) Eeg based emotion recognition by combining functional connectivity network and local activations. IEEE Trans Biomed Eng 66(10):2869–2881

    Google Scholar 

  38. Safarbali B, Golpayegani SMRH (2019) Nonlinear dynamic approaches to identify atrial fibrillation progression based on topological methods. Biomed Signal Process Control 53:101563

    Google Scholar 

  39. Khasawneh FA, Munch E, Perea JA (2018) Chatter classification in turning using machine learning and topological data analysis. IFAC-PapersOnLine 51(14):195–200

    Google Scholar 

  40. Chung MK, Hanson JL, Lee H, Adluru N, Alexander AL, Davidson RJ, Pollak SD (2013) Persistent homological sparse network approach to detecting white matter abnormality in maltreated children: Mri and dti multimodal study. In: Medical image computing and computer-assisted intervention–MICCAI 2013: 16th international conference, Nagoya, Japan, September 22–26, 2013, Proceedings, Part I 16, pp 300–307. Springer

  41. Sizemore AE, Giusti C, Kahn A, Vettel JM, Betzel RF, Bassett DS (2018) Cliques and cavities in the human connectome. J Computat Neurosci 44:115–145

    MathSciNet  Google Scholar 

  42. Gao X, Cao H, Ming D, Qi H, Wang X, Wang X, Chen R, Zhou P (2014) Analysis of eeg activity in response to binaural beats with different frequencies. Int J Psychophysiol 94(3):399–406

    Google Scholar 

  43. Zheng R, Wang Z, He Y, Zhang J (2022) Eeg-based brain functional connectivity representation using amplitude locking value for fatigue-driving recognition. Cognit Neurodyn, pp 1–12

  44. SA Nasab , Panahi S, Ghassemi F, Jafari S, Rajagopal K Ghosh D, Perc M (2021) Functional neuronal networks reveal emotional processing differences in children with adhd. Cognit Neurodyn, pp 1–10

  45. Piqueira JRC (2011) Network of phase-locking oscillators and a possible model for neural synchronization. Commun Nonlinear Sci Numer Simul 16(9):3844–3854

    MathSciNet  Google Scholar 

  46. Aktas ME, Akbas E, El Fatmaoui A (2019) Persistence homology of networks: methods and applications. Appl Netw Sci 4(1):1–28

    Google Scholar 

  47. Attali D, Lieutier A, Salinas D (2011) Vietoris-rips complexes also provide topologically correct reconstructions of sampled shapes. In: Proceedings of the twenty-seventh annual symposium on computational geometry, pp 491–500

  48. Zomorodian A (2010) Fast construction of the vietoris-rips complex. Comput Graphics 34(3):263–271

    Google Scholar 

  49. Tauzin G, Lupo U, Tunstall L, Pérez JB, Caorsi M, Medina-Mardones AM, Dassatti A, Hess K (2021) giotto-tda: a topological data analysis toolkit for machine learning and data exploration. J Mach Learn Res 22(1):1834–1839

    MathSciNet  Google Scholar 

  50. Cohen-Steiner D, Edelsbrunner H, Harer J (2005) Stability of persistence diagrams. In: Proceedings of the twenty-first annual symposium on Computational geometry, pp 263–271

  51. Bubenik P, Dłotko P (2017) A persistence landscapes toolbox for topological statistics. J Symbol Comput 78:91–114

    MathSciNet  Google Scholar 

  52. Rucco M, Castiglione F, Merelli E, Pettini M (2016) Characterisation of the idiotypic immune network through persistent entropy. In: Proceedings of ECCS 2014: European conference on complex systems, pp 117–128. Springer

  53. Garin A, Tauzin G (2019) A topological" reading" lesson: classification of mnist using tda. In: 2019 18th IEEE international conference on machine learning and applications (ICMLA), pp 1551–1556. IEEE

  54. Song T, Zheng W, Song P, Cui Z (2018) Eeg emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput 11(3):532–541

    Google Scholar 

  55. Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2011) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31

    Google Scholar 

  56. Katsigiannis S, Ramzan N (2017) Dreamer: a database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Informat 22(1):98–107

    Google Scholar 

  57. Zheng W-L, Bao-Liang L (2015) Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks. IEEE Trans Autonom Mental Develop 7(3):162–175

    Google Scholar 

  58. Duan R-N, Zhu J-Y, Lu B-L (2013) Differential entropy feature for eeg-based emotion classification. In: 2013 6th International IEEE/EMBS conference on neural engineering (NER), pp 81–84. IEEE

  59. Zhang Y, Yan G, Chang W, Huang W, Yuan Y (2023) Eeg-based multi-frequency band functional connectivity analysis and the application of spatio-temporal features in emotion recognition. Biomed Signal Process Control 79:104157

    Google Scholar 

  60. Wang W (2021) Brain network features based on theta-gamma cross-frequency coupling connections in eeg for emotion recognition. Neurosci Lett 761:136106

    Google Scholar 

  61. Chao H, Liu Y (2020) Emotion recognition from multi-channel eeg signals by exploiting the deep belief-conditional random field framework. IEEE Access 8:33002–33012

    Google Scholar 

  62. Atkinson J, Campos D (2016) Improving bci-based emotion recognition by combining eeg feature selection and kernel classifiers. Exp Syst Appl 47:35–41

    Google Scholar 

  63. Xu H, Wang X, Li W, Wang H, Bi Q (2019) Research on eeg channel selection method for emotion recognition. In: 2019 IEEE international conference on robotics and biomimetics (ROBIO), pp 2528–2535. IEEE

  64. Cui G, Li X, Touyama H (2023) Emotion recognition based on group phase locking value using convolutional neural network. Sci Rep 13(1):3769

    Google Scholar 

  65. Wang Y, Ombao H, Chung MK (2019) Statistical persistent homology of brain signals. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1125–1129. IEEE

  66. Majumder S, Apicella F, Muratori F, Das K (2020) Detecting autism spectrum disorder using topological data analysis. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1210–1214. IEEE

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 61373116, in part by the National Natural Science Foundation of China under Grant 62002287, and in part by the Shaanxi Provincial Key Research and Development Project under Grant 2022SF-037.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, Xi’an, 710121, Shaanxi, China

    Zhong-min Wang, Sha Li, Jie Zhang & Chen Liang

  2. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an University of Posts and Telecommunications, Xi’an, 710121, Shaanxi, China

    Zhong-min Wang, Jie Zhang & Chen Liang

  3. Xi’an Key Laboratory of Big Data and Intelligent Computing, Xi’an, 710121, Shaanxi, China

    Zhong-min Wang, Jie Zhang & Chen Liang

Authors
  1. Zhong-min Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sha Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZW and SL developed the idea of the study, participated in its design and coordination, and helped to draft the manuscript. JZ and CL contributed to the acquisition and interpretation of data.

Corresponding author

Correspondence to Sha Li.

Ethics declarations

Conflict of interest

No conflict of interest.

Ethics approval

All authors approve.

Consent to participate

All authors agree.

Consent for publication

All authors read and approved the final manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Zm., Li, S., Zhang, J. et al. Emotion recognition based on phase-locking value brain functional network and topological data analysis. Neural Comput & Applic 36, 7903–7922 (2024). https://doi.org/10.1007/s00521-024-09479-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09479-3

Keywords

Search

Navigation