Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

A predictive speller controlled by a brain-computer interface based on motor imagery

Published: 25 October 2012 Publication History

Abstract

Persons suffering from motor disorders have limited possibilities for communicating and normally require assistive technologies to fulfill this primary need. Promising means of providing basic communication abilities to subjects affected by severe motor impairments include brain-computer interfaces (BCIs), that is, systems that directly translate brain signals into device commands, bypassing any muscle or nerve mediation. To date, the use of BCIs for effective verbal communication is yet an open issue, primarily due to the low rates of information transfer that can be achieved with this technology. Still, performance of BCI spelling applications could be considerably improved by a smart user interface design and by the adoption of natural language processing (NLP) techniques for text prediction. The objective of this work is to suggest an approach and a user interface for BCI spelling applications combining state-of-the-art BCI and NLP techniques to maximize the overall communication rate of the system. The BCI paradigm adopted is motor imagery, that is, when the subject imagines moving a certain part of the body, he/she produces modifications to specific brain rhythms that are detected in real-time through an electroencephalogram and translated into commands for a spelling application. By maximizing the overall communication rate, our approach is twofold: on one hand, we maximize the information transfer rate from the control signal, on the other hand, we optimize the way this information is employed for the purpose of verbal communication. The achieved results are satisfactory and comparable with the latest works reported in literature on motor-imagery BCI spellers. For the three subjects tested, we obtained a spelling rate of respectively 3 char/min, 2.7 char/min, and 2 char/min.

Supplementary Material

a20-dalbis-apndx.pdf (dalbis.zip)
Supplemental movie, appendix, image and software files for, A predictive speller controlled by a brain-computer interface based on motor imagery.

References

[1]
Bauer, G., Gerstenbrand, F., and Rumpl, E. 1979. Varieties of the locked-in syndrome. J. Neurol. 221, 2, 77--91.
[2]
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., and Flor, H. 1999. A spelling device for the paralysed. Nature 398, 6725, 297--298.
[3]
Bishop, C. 2006. Pattern Recognition and Machine Learning. Springer, New York.
[4]
Blankertz, B., Dornhege, G., Krauledat, M., Schröder, M., Williamson, J., Murray-Smith, R., and Müller, K. 2006. The berlin brain-computer interface presents the novel mental typewriter hex-o-spell. In Proceedings of the 3rd International Brain-Computer Interface Workshop and Training Course. 108--109.
[5]
Blankertz, B., Krauledat, M., Dornhege, G., Williamson, J., Murray-Smith, R., and Muller, K. 2007. A note on brain actuated spelling with the Berlin Brain-Computer Interface. Lecture Notes in Computer Science, vol. 4555, Springer, Berlin, 759.
[6]
Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., and Muller, K. 2008. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Proces. Mag., 25, 1, 41--56.
[7]
Burg, J. 1975. Maximum Entropy Spectral Analysis. Stanford University. http://sep.www.stanford.edu/theses/sep06.
[8]
Cohen, J. 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 1, 37.
[9]
Curran, E. and Stokes, M. 2003. Learning to control brain activity: A review of the production and control of EEG components for driving brain-computer interface (bci) systems. Brain Cogn. 51, 3, 326--336.
[10]
Dal Seno, B., Matteucci, M., and Mainardi, L. T. 2010a. Online detection of P300 and error potentials in a bci speller. Comput. Intell. Neurosc. 2010, Article ID 307254.
[11]
Dal Seno, B., Matteucci, M., and Mainardi, L. T. 2010b. The utility metric: A novel method to assess the overall performance of discrete brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 1, 20--28.
[12]
Donchin, E., Spencer, K., and Wijesinghe, R. 2000. The mental prosthesis: Assessing the speed of a P300-based brain-computer interface. IEEE Trans. Rehabil. Eng. 8, 2, 174--179.
[13]
Farwell, L. and Donchin, E. 1988. Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysi. 70, 6, 510--523.
[14]
Fukunaga, K. 1990. Introduction to Statistical Pattern Recognition. Academic Press, Waltham, MA.
[15]
Gale, W. 1995. Good-Turing smoothing without tears. J. Quant. Ling.
[16]
Goldberg, D. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Boston, MA.
[17]
Graimann, B., Allison, B., and Pfurtscheller, G. 2011. Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction. Springer Verlag, Berlin.
[18]
Holland, J. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press, Cambridge, MA.
[19]
Hopcroft, J., Motwani, R., and Ullman, J. 2007. Introduction to Automata Theory, Languages, and Computation 3rd Ed. Prentice Hall, Upper Saddle River, NJ.
[20]
Jansen, B. H., Bourne, J. R., and Ward, J. W. 1981. Autoregressive estimation of short segment spectra for computerized EEG analysis. IEEE Tran. Biomed. Eng. BME-28, 9, 630--638.
[21]
Jasper, H. 1958. The 10/20 international electrode system. EEG Clin. Neurophysio. 10, 371--375.
[22]
Jeannerod, M. 1995. Mental imagery in the motor context. Neuropsychologia 33, 11, 1419--1432.
[23]
Jurafsky, D. and Martin, J. H. 2000. Speech and Language Processing. Prentice Hall, Upper Saddle River, NJ.
[24]
Kalcher, J., Flotzinger, D., Neuper, C., Gölly, S., and Pfurtscheller, G. 1996. Graz brain-computer interface II: Towards communication between humans and computers based on online classification of three different EEG patterns. Med. Biol. Eng. Comput. 34, 5, 382--388.
[25]
Katz, S. 1987. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Trans. Acous. Speech Signal Proces. 35, 3, 400--401.
[26]
Koles, Z. 1991. The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalogr. Clin. Neurophysi. 79, 6, 440--447.
[27]
Kozelka, J. and Pedley, T. 1990. Beta and mu rhythms. J. Clin. Neurophysiol. 7, 2, 191--208.
[28]
Krusienski, D., McFarland, D., and Wolpaw, J. 2006. An evaluation of autoregressive spectral estimation model order for brain-computer interface applications. In Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS'06). 1323--1326.
[29]
Leech, G. 1992. 100 million words of English: The British National Corpus. Lang. Res. 28, 1, 1--13.
[30]
Leech, G., Rayson, P., and Wilson, A. 2001. Word Frequencies in Written and Spoken English: Based on the British National Corpus. Longman, London.
[31]
Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., and Arnaldi, B. 2007. A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 2, R1--R13.
[32]
Manning, C. and Schütze, H. 1999. Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge, MA.
[33]
McFarland, D., Krusienski, D., Sarnacki, W., and Wolpaw, J. 2008. Emulation of computer mouse control with a noninvasive brain--computer interface. J. Neural eng. 5, 101.
[34]
McFarland, D., Lefkowicz, A., and Wolpaw, J. 1997. Design and operation of an EEG-based brain-computer interface with digital signal processing technology. Behav. Res. Methods 29, 3, 337--345.
[35]
McFarland, D., McCane, L., David, S., and Wolpaw, J. 1997. Spatial filter selection for EEG-based communication. Electroencephalogr. Clin. Neurophysiol. 103, 3, 386--394.
[36]
McLachlan, G. 2004. Discriminant Analysis and Statistical Pattern Recognition. Wiley-IEEE, Press, Hoboken, NJ.
[37]
Migliore, D., Blatt, R., Matteucci, M., Ceriani, S., Fontana, G., and Seno, B. D. 2008. Brain control of a smart wheelchair. In Proceedings of International Conference on Inteligent Autonomous Systems. 221--228.
[38]
Neuper, C., Müller, G., Kübler, A., Birbaumer, N., and Pfurtscheller, G. 2003. Clinical application of an EEG-based brain--computer interface: A case study in a patient with severe motor impairment. Clin. Neurophysio. 114, 3, 399--409.
[39]
Neuper, C., Schlögl, A., and Pfurtscheller, G. 1999. Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. J. Clin. Neurophysiol. 16, 4, 373.
[40]
Obermaier, B., Muller, G., and Pfurtscheller, G. 2003. “Virtual keyboard” controlled by spontaneous EEG activity. IEEE Trans. Neural Sys. Rehabil. Eng. 11, 4, 422--426.
[41]
Pardey, J., Roberts, S., and Tarassenko, L. 1996. A review of parametric modelling techniques for EEG analysis. Med. Eng. Physics 18, 1, 2--11.
[42]
Pfurtscheller, G. and Lopes da Silva, F. 1999. Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clin. Neurophysiol. 110, 11, 1842--1857.
[43]
Pfurtscheller, G. and Neuper, C. 1997. Motor imagery activates primary sensorimotor area in humans. Neurosci. Lett. 239, 2--3, 65--68.
[44]
Pfurtscheller, G., Neuper, C., Muller, G., Obermaier, B., Krausz, G., Schlogl, A., Scherer, R., Graimann, B., Keinrath, C., Skliris, D. Wörtz, M., Supp, G., and Schrank. C. 2003. Graz-BCI: state of the art and clinical applications. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 2, 1--4.
[45]
Pierce, J. 1980. An Introduction to Information Theory. Dover Publications, Mineola, NY.
[46]
Plum, F. and Posner, J. 1966. The Diagnosis of Stupor and Coma. Oxford University Press, New York, NY.
[47]
Polich, J. 1997. On the relationship between EEG and P300: Individual differences, aging, and ultradian rhythms. Int. J. Psychophysiol. 26, 1--3, 299--317.
[48]
Pregenzer, M. and Pfurtscheller, G. 1999. Frequency component selection for an EEG-based brain to computer interface. IEEE Trans. Rehabil. Eng. 7, 4, 413--419.
[49]
Ramoser, H., Müller-Gerking, J., and Pfurtscheller, G. 2000. Optimal spatial filtering of single trial EEG during imagined handmovement. IEEE Trans. Rehabil. Eng. 8, 4, 441--446.
[50]
Ryan, D., Frye, G., Townsend, G., Berry, D., Mesa-G, S., Gates, N., and Sellers, E. 2011. Predictive spelling with a P300-based brain--computer interface: Increasing the rate of communication. Int. J. Hum.-Comput. Interac. 27, 1, 69--84.
[51]
Schalk, G., McFarland, D., Hinterberger, T., Birbaumer, N., and Wolpaw, J. 2004. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 6, 1034--1043.
[52]
Scherer, R., Müller, G., Neuper, C., Graimann, B., and Pfurtscheller, G. 2004. An asynchronously controlled EEG-based virtual keyboard: Improvement of the spelling rate. IEEE Trans. Biomed. Eng. 51, 6, 979--984.
[53]
Spyers-Ashby, J., Bain, P., and Roberts S. J. 1998. A comparison of fast Fourier transform (FFT) and autoregressive (AR) spectral estimation techniques for the analysis of tremor data. J. Neurosc. Methods 83, 1, 35--43.
[54]
Wang, C., Guan, C., and Zhang, H. 2005. P300 brain-computer interface design for communication and control applications. In Proceedings of the 27th IEEE-EMBS Annual International Conference of the Engineering in Medicine and Biology Society. 5400--5403.
[55]
Wang, T., Deng, J., and He, B. 2004. Classifying EEG-based motor imagery tasks by means of time--frequency synthesized spatial patterns. Clin. Neurophysiol. 115, 12, 2744--2753.
[56]
Wolpaw, J., Birbaumer, N., McFarland, D., Pfurtscheller, G., and Vaughan, T. 2002. Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 6, 767--791.
[57]
Wolpaw, J. and McFarland, D. 2004. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. In Proceedings of the National Academy of Sciences of the United States of America 101, 51, 17849.
[58]
Wolpaw, J., McFarland, D., Neat, G., and Forneris, C. 1991. An EEG-based brain-computer interface for cursor control. Electroencephalog. Clin. Neurophysiol. 78, 3, 252--259.
[59]
Wolpaw, J., McFarland, D., and Vaughan, T. 2000. Brain-computer interface research at the Wadsworth Center. IEEE Trans. Rehabil. Eng. 8, 2, 222--226.
[60]
Wonnacott, T. and Wonnacott, R. 1985. Introductory Statistics. Wiley, New York, NY.
[61]
Yan, T., Jingtian, T., and Andong, G. 2008. Multi-class EEG classification for brain computer interface based on CSP. In Proceedings of the BMEI International Conference on BioMedical Engineering and Informatic. 469--472.

Cited By

View all
  • (2023)Brain-Computer InterfaceHumans and Electricity10.1007/978-3-031-20784-6_10(223-248)Online publication date: 4-Apr-2023
  • (2022)Classification Strategies for P300-Based BCI-Spellers Adopting the Row Column ParadigmSensors10.3390/s2223915922:23(9159)Online publication date: 25-Nov-2022
  • (2022)Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and ReuseACM Transactions on Computer-Human Interaction10.1145/349055429:4(1-43)Online publication date: 31-Mar-2022
  • Show More Cited By

Index Terms

  1. A predictive speller controlled by a brain-computer interface based on motor imagery

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Computer-Human Interaction
    ACM Transactions on Computer-Human Interaction  Volume 19, Issue 3
    October 2012
    209 pages
    ISSN:1073-0516
    EISSN:1557-7325
    DOI:10.1145/2362364
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 October 2012
    Accepted: 01 April 2012
    Revised: 01 September 2011
    Received: 01 November 2010
    Published in TOCHI Volume 19, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. BCI
    2. BMI
    3. NLP
    4. brain-computer interface
    5. speller
    6. text prediction

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)42
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 15 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Brain-Computer InterfaceHumans and Electricity10.1007/978-3-031-20784-6_10(223-248)Online publication date: 4-Apr-2023
    • (2022)Classification Strategies for P300-Based BCI-Spellers Adopting the Row Column ParadigmSensors10.3390/s2223915922:23(9159)Online publication date: 25-Nov-2022
    • (2022)Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and ReuseACM Transactions on Computer-Human Interaction10.1145/349055429:4(1-43)Online publication date: 31-Mar-2022
    • (2021)Brain–Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and PerformanceBrain Sciences10.3390/brainsci1104045011:4(450)Online publication date: 1-Apr-2021
    • (2021)Direct-Sense Brain–Computer Interfaces and Wearable ComputersIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2020.304138251:1(298-312)Online publication date: Jan-2021
    • (2020)Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEGSensors10.3390/s2016462920:16(4629)Online publication date: 17-Aug-2020
    • (2020)A Review of Brain Computer Interface Spellers2020 International Conference on Emerging Trends in Smart Technologies (ICETST)10.1109/ICETST49965.2020.9080743(1-6)Online publication date: Mar-2020
    • (2019)Stroke-Gesture Input for People with Motor ImpairmentsProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300445(1-14)Online publication date: 2-May-2019
    • (2019)A Novel Real-Time Multi-Phase BCI Speller Based on Sliding Control Paradigm of SSVEPIEEE Access10.1109/ACCESS.2019.29416427(133974-133981)Online publication date: 2019
    • (2019)Optimising non-invasive brain-computer interface systems for free communication between naïve human participantsScientific Reports10.1038/s41598-019-55166-y9:1Online publication date: 10-Dec-2019
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media