Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Brain-Computer Interface System Based on P300 Processing with Convolutional Neural Network, Novel Speller, and Low Number of Electrodes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The P300 wave has been successfully employed to develop brain-computer interfaces (BCI) for speller applications. However, methods to analyze the P300 require computers with high processing capability because they are computationally complex and require many electrodes. Therefore, this paper proposes a novel BCI speller system based on the P300 wave that employs a few electrodes and a processing method aimed to design ubiquitous and embedded applications. The experiments were developed with a dataset generated by our BCI data acquisition system. The BCI speller developed requires five electrodes for data acquisition, and the visual interface is an improved Donchin speller. Our BCI includes a novel processing method composed of the following modules: preprocessing, signal averaging, low computational cost convolutional neural network, and character prediction. The network has two feature extraction sections, a fully connected layer and a SoftMax layer. According to the results, the proposed BCI speller has an accuracy of 96% using just five electrodes, and it is similar to the best BCI for P300 analysis described in the literature. The processing time makes the system practical for online applications since the processing method has a low computational burden and the acquisition system has the lowest number of electrodes for P300 analysis reported in the literature. Considering the low computational burden, the low number of electrodes required, and the accuracy achieved, we conclude that our proposed BCI speller may be considered as one of the best spellers based on P300.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Payne JR, Baell O, Geddes H, Fitzgibbon B, Emonson M, Hill AT, et al. Experienced meditators exhibit no differences to demographically matched controls in theta phase synchronization, P200, or P300 during an auditory oddball task. Mindfulness. 2020;11:643–59.

  2. Li F, Yi C, Jiang Y, Liao Y, Si Y, Dai J, et al. Different contexts in the oddball paradigm induce distinct brain networks in generating the P300. Front Hum Neurosci. 2019;12:1–10.

    Google Scholar 

  3. Martinez-Cagigal V, Gomez-Pilar J, Alvarez D, Hornero R. An asynchronous P300-based brain-computer interface web browser for severely disabled people. IEEE Trans Neural Syst Rehabil Eng. 2017;25:1332–42.

    Google Scholar 

  4. Guy V, Soriani MH, Bruno M, Papadopoulo T, Desnuelle C, Clerc M. Brain computer interface with the P300 speller: usability for disabled people with amyotrophic lateral sclerosis. Ann Phys Rehabil Med. 2018;61:5–11.

    Google Scholar 

  5. De Venuto D, Annese VF, Mezzina G. Real-time P300-based BCI in mechatronic control by using a multi-dimensional approach. IET Softw. 2018;12:418–24.

    Google Scholar 

  6. Zhao W, Zhang X, Qu J, Xiao J, Huang Y. A virtual smart home based on EEG control. In: Proc 2019 IEEE 9th Int Conf Electron Inf Emerg Commun; 2019. p. 85–9.

    Google Scholar 

  7. Tjandrasa H, Djanali S. Classification of P300 event-related potentials using wavelet transform, MLP, and soft margin SVM. In: Proc - 2018 10th Int Conf Adv Comput Intell. Piscataway: IEEE; 2018. p. 343–7.

    Google Scholar 

  8. Bhatnagar V, Yede N, Keram RS, Chaurasiya RK. A modified approach to ensemble of SVM for P300 based brain computer interface. In: 2016 Int Conf Adv Hum Mach Interact. p. 2016.

  9. Rakotomamonjy A, Guigue V. BCI competition III: dataset II- ensemble of SVMs for BCI P300 speller. IEEE Trans Biomed Eng. 2008;55:1147–54.

    Google Scholar 

  10. Cecotti H, Gr A. Convolutional neural networks for P300 detection with application to brain-computer interfaces. IEEE Trans Pattern Anal Mach Intell. 2011;33:433–45.

    Google Scholar 

  11. Liu M, Wu W, Gu Z, Yu Z, Qi FF, Li Y. Deep learning based on batch normalization for P300 signal detection. Neurocomputing. Elsevier B.V. 2018;275:288–97.

    Google Scholar 

  12. Nashed NN, Eldawlatly S, Aly GM. A deep learning approach to single-trial classification for P300 spellers. In: Middle East Conf Biomed Eng MECBME. Piscataway: IEEE; 2018. p. 11–6.

    Google Scholar 

  13. Xu N, Gao X, Hong B, Miao X, Gao S, Yang F. BCI competition 2003 - data set IIb: enhancing P300 wave detection using ICA-based subspace projections for BCI applications. IEEE Trans Biomed Eng. 2004;51:1067–72.

    Google Scholar 

  14. Wang Y, Shen J, Liang J, Ji Y. Research of P300 feature extraction algorithm based on ICA and wavelet transform. In: Proc - 2014 6th Int Conf Intell Human-Machine Syst Cybern; 2014. p. 41–5.

    Google Scholar 

  15. Liang N, Bougrain L. Averaging techniques for single-trial analysis of oddball event-related potentials. In: 4th Int Brain-Computer Interface Work; 2008. p. 1–6.

    Google Scholar 

  16. Blankertz OB. Documentation Wadsworth BCI. Dataset (P300 Evoked Potentials) BCI competition III challenge. Interface. 2004. http://www.bbci.de/competition/ii/. Accessed 17 June 2020.

  17. Aydin EA, Bay OF, Guler I. P300-based asynchronous brain computer interface for environmental control system. IEEE J Biomed Heal Informatics. IEEE. 2018;22:653–63.

    Google Scholar 

  18. Oralhan Z. A new paradigm for region-based P300 speller in brain computer interface. IEEE Access. IEEE. 2019;7:106618–27.

    Google Scholar 

  19. Symeonidou ER, Nordin AD, Hairston WD, Ferris DP. Effects of cable sway, electrode surface area, and electrode mass on electroencephalography signal quality during motion. Sensors (Switzerland). 2018;18:1–13.

    Google Scholar 

  20. Melnik A, Legkov P, Izdebski K, Kärcher SM, Hairston WD, Ferris DP, et al. Systems, subjects, sessions: to what extent do these factors influence EEG data? Front Hum Neurosci. 2017;11:1–20.

    Google Scholar 

  21. Guttmann-Flury E, Sheng X, Zhang D, Zhu X. A priori sample size determination for the number of subjects in an EEG experiment. In: Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS: IEEE; 2019. p. 5180–3.

  22. McCrimmon CM, Fu JL, Wang M, Lopes LS, Wang PT, Karimi-Bidhendi A, et al. Performance assessment of a custom, portable, and low-cost brain-computer interface platform. IEEE Trans Biomed Eng. 2017;64:2313–20.

    Google Scholar 

  23. Neumann T, Baum AK, Baum U, Deike R, Feistner H, Scholz M, et al. Assessment of the technical usability and efficacy of a new portable dry-electrode EEG recorder: first results of the HOMEONE study. Clin Neurophysiol. 2019;130:2076–87.

    Google Scholar 

  24. Albasri A, Abdali-Mohammadi F, Fathi A. EEG electrode selection for person identification thru a genetic-algorithm method. J Med Syst. 2019;43:297.

    Google Scholar 

  25. Goshvarpour A, Goshvarpour A. A novel approach for EEG electrode selection in automated emotion recognition based on Lagged Poincare’s Indices and sLORETA. Cognit Comput. 2020;12:602–18.

  26. Lin Z, Zhang C, Zeng Y, Tong L, Yan B. A novel P300 BCI speller based on the Triple RSVP paradigm. Sci Rep. Springer US. 2018;8:1–9.

    Google Scholar 

  27. Farwell L, Donchin F. Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1998;70:510–23.

    Google Scholar 

  28. Larrivee D. In: Larrivve D, editor. Envolving BCI therapy: engaging brain state dynamics. First Edit. London: Interchopen Limited; 2018.

    Google Scholar 

  29. Ratcliffe L, Puthusserypady S. Importance of graphical user interface in the design of P300 based brain–computer interface systems. Comput Biol Med. 2020;117:103599.

    Google Scholar 

  30. Brysbaert M. How many words do we read per minute? A review and meta-analysis of reading rate. J Mem Lang. Elsevier. 2019;109:1–94.

    Google Scholar 

  31. Folstein JR, Monfared SS. Extended categorization of conjunction object stimuli decreases the latency of attentional feature selection and recruits orthography-linked ERPs. Cortex. 2019;120:49–65.

    Google Scholar 

  32. Chang S. Nam, Anton Nijholt FL. Brain-Computer Interfaces Handbook: Technological and Theoretical Advances. Appl. Phys. A. 2018.

    Google Scholar 

  33. Overbye K, Huster RJ, Walhovd KB, Fjell AM, Tamnes CK. Development of the P300 from childhood to adulthood: a multimodal EEG and MRI study. Brain Struct Funct. Springer. Berlin Heidelberg. 2018;223:4337–49.

    Google Scholar 

  34. Di Russo F, Berchicci M, Bianco V, Perri RL, Pitzalis S, Quinzi F, et al. Normative event-related potentials from sensory and cognitive tasks reveal occipital and frontal activities prior and following visual events. Neuroimage. Elsevier Inc. 2019;196:173–87.

    Google Scholar 

  35. Shen J, Liang J, Shi J, Wang Y. A dynamic submatrix-based P300 online brain-computer interface. Biomed Signal Process Control. 2015;15:27–32.

    Google Scholar 

  36. Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, et al. How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett. 2009;462:94–8.

    Google Scholar 

  37. Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J Neurosci Methods. 2008;167:15–21.

    Google Scholar 

  38. Tang Y, Wang J, Zhang T, Xu L, Qian Z, Cui H, et al. P300 as an index of transition to psychosis and of remission: data from a clinical high risk for psychosis study and review of literature. Schizophr Res. 2019;1–10. https://doi.org/10.1016/j.schres.2019.02.014.

  39. Zazio A, Schreiber M, Miniussi C, Bortoletto M. Modelling the effects of ongoing alpha activity on visual perception: the oscillation-based probability of response. Neurosci Biobehav Rev. 2020;112:242–53.

    Google Scholar 

  40. Stehlin SAF, Nguyen XP, Niemz MH. EEG with a reduced number of electrodes: where to detect and how to improve visually, auditory and somatosensory evoked potentials. Biocybern Biomed Eng. Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences. 2018;38:700–7.

    Google Scholar 

  41. Fernández-Rodríguez Á, Medina-Juliá MT, Velasco-Álvarez F, Ron-Angevin R. Effects of spatial stimulus overlap in a visual P300-based brain-computer interface. J Pre-proofs. 2020;431:134–42.

    Google Scholar 

  42. Tenssay F, Wang H. Analysis of EEG signals during visual processing: An ERP study. In: 2019 IEEE Int Conf Signal Process Commun Comput: ICSPCC; 2019. p. 1–5.

  43. Palejwala AH, O’Connor KP, Pelargos P, Briggs RG, Milton CK, Conner AK, et al. Anatomy and white matter connections of the lateral occipital cortex. Surg Radiol Anat. Springer Paris. 2019;1:1–14.

    Google Scholar 

  44. Lizhe T, Jean J. Digital Signal Processing: Fundamentals and Applications. 3rd ed. Amsterdam: Elsevier; 2018.

    Google Scholar 

  45. Dickson DS, Wicha NYY. P300 amplitude and latency reflect arithmetic skill: an ERP study of the problem size effect. Biol Psychol. Elsevier. 2019;148:1–17.

    Google Scholar 

  46. Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognit Comput. 2018;10:368–80.

    Google Scholar 

  47. Haridas R, Jyothi RL. Convolutional neural networks: a comprehensive survey. Int J Appl Eng Res. 2019;14:780–9.

    Google Scholar 

  48. Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recognit. 2018;77:354–77.

    Google Scholar 

  49. Zheng J, Cao X, Zhang B, Zhen X, Su X. Deep ensemble machine for video classification. IEEE Trans Neural Networks Learn Syst. IEEE. 2019;30:553–65.

    Google Scholar 

  50. Khan ZA, Zubair S, Alquhayz H, Azeem M, Ditta A. Design of momentum fractional stochastic gradient descent for recommender systems. IEEE Access. IEEE. 2019;7:179575–90.

    Google Scholar 

  51. Solé-Casals J, Caiafa CF, Zhao Q, Cichocki A. Brain-computer interface with corrupted EEG data: a tensor completion approach. Cognit Comput. 2018;10:1062–74.

    Google Scholar 

  52. Berrar D. Cross-Validation. In: Encycl Bioinforma Comput Biol; 2019. p. 542–5.

    Google Scholar 

  53. Cintra RJ, Duffner S, Garcia C, Leite A. Low-complexity approximate convolutional neural networks. IEEE Trans Neural Networks Learn Syst. 2018;29:5981–92.

    Google Scholar 

Download references

Acknowledgments

The authors thank the volunteers that participated in the dataset elaboration.

Funding

This study was funded by Tecnológico Nacional de México (TecNM) under grant no. 7598.20-P.

Author information

Authors and Affiliations

  1. Visual Perception Applications on Robotics Lab, Tecnológico Nacional de México/I.T. Chihuahua, Av. Tecnológico 2909, 31130, Chihuahua, México

    Juan A. Ramirez-Quintana, Luis Madrid-Herrera & Mario I. Chacon-Murguia

  2. LASIX Lab, Tecnológico Nacional de México/I.T. Chihuahua, Av. Tecnológico 2909, 31130, Chihuahua, México

    Luis F. Corral-Martinez

Authors
  1. Juan A. Ramirez-Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis Madrid-Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario I. Chacon-Murguia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luis F. Corral-Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan A. Ramirez-Quintana.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This research involved human volunteers and was developed under safe conditions. The equipment used for experiments was safe and noninvasive for the subjects. All procedures performed in this research were in accordance with the ethical standards of the Tecnológico Nacional de México, the Mexican norm NOM-012-SSA3-2012, and the 1964 Helsinki declarations and its amendments.

This article does not contain any studies with animals performed by any of the authors.

Informed Consent

All the volunteer subjects signed the consent to participate in this research, to publish their EEG signals and to publish the results of the experiments with their EEG signals.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramirez-Quintana, J.A., Madrid-Herrera, L., Chacon-Murguia, M.I. et al. Brain-Computer Interface System Based on P300 Processing with Convolutional Neural Network, Novel Speller, and Low Number of Electrodes. Cogn Comput 13, 108–124 (2021). https://doi.org/10.1007/s12559-020-09744-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-020-09744-2

Keywords

Search

Navigation