Brain-Computer Interface Review

Szilárd BULÁRKA Aurel GONTEAN

Applied Electronics Department Applied Electronics Department
University of Politehnica Timisoara University of Politehnica Timisoara
Timisoara, Timis Timisoara, Timis
szilard.bularka@student.upt.ro aurel.goneant@upt.ro

Abstract— Brain-Computer Interface (BCI) is a device that silicon sensor placement for ECoG, fMRI play a key role.
allows direct communication path between central nervous Todays BCI requirements are: good temporal and spatial
system and external devices without peripheral nerves resolution, mobility, user friendliness, ethical, discreteness,
dependency. Brain-Computer Interface and its applications robustness. fMRI and fNIR are used mainly as research tools
reached beyond medical applications, it is used to enhance, because the imaging process of those is time-consuming and
improve, restore or replace functions or it can be used as a the mobility of the user is highly restricted. For modern BCIs
research tool. Stakeholders of the field have developed future EEG and ECoG offers optimal performances and mobility.
plans and the ethical in research. the multidisciplinary nature of ECoG is a highly invasive method and involve higher costs
Brain-Computer Interface is based on neuroscience, signal
but the silicone based sensors offer higher signal to noise ratio,
processing, biomedical sensors, and hardware. This paper aims
to present briefly the current state and the future of Brain-
better spatial resolution, ease of use and most of them offer
Computer Interfaces. The future works of the author are based discreteness. ECoG is mainly used in replace and restore
on hardware approach using System-On-Chip combined with applications. Thanks to its noninvasiveness EEG is preferred
novelty memristor crossbar memory, which can be used to for most of the applications, nowadays dry electrodes offer
emulate axons and neurons directly in the hardware. ease of use and discreteness. Using novelty signal processing
techniques the performance of EEG systems has been
Keywords—Brain-Computer Interface,BCI, state of the art, BCI increased in the last years. The appearance of medical graded
applications, neuroscience, computational neuroscience gadget EEG headsets like EPOC by Emotiv, Muse by
Interaxon, etc. have opened a new application area in the field
I. INTRODUCTION of entertainment and mental state monitoring for BCI. The
reduced cost and the performances offered by most of these
The brain-computer interface is a device that allows direct devices offer the possibility to young scientists to contribute to
communication path between the brain and external devices research. Because the field of BCI research is a young and
without peripheral nerves dependency. .In the early days BCI growing, BNCI Horizon 2020 [1] is a Coordination and
was more a science fiction topic, nowadays thanks to the huge Support Action founded within the European Commission's
advancements in neuroscience, signal processing, Framework Programme 7 to facilitate the communication and
computational neuroscience, sensors etc. BCI is a growing collaboration between stakeholders. BNCI Horizon 2020 have
area of research. Since Hans Berger has recorded the neuron proposed a roadmap which includes the current state and a
firing process in 1924 using electroencephalography, (EEG) guideline for future applications. The BCI Awards [2] donated
unlocked the possibility to investigate schizophrenia and other by g.tec offer a good overview of outstanding and innovative
brain diseases. Nowadays among EEG we can monitor the research projects in the field of BCI research. On the other
brain waves using various novelty noninvasive methods like hand, the BCI Awards offer a good dataset for benchmarking.
Functional Magnetic Resonance Imaging (fMRI), Functional
Near-infrared Spectroscopy Imaging (fNIR) and invasive
methods like Electrocorticography (ECoG). In the early days
of BCI, the main applications were focused on motor
rehabilitation. Nowadays BCI is used beyond medical
applications like entertainment, military, research tool, etc.
The reasoning why most of the BCI applications are motor
related is strongly correlated why animals need brains.
According to Daniel Wolpert, the need for a brain is related to
movement. The studies of squids can argue with D. Wolpert's
Figure. 1. Emotiv EPOC 16 chanel EEG headset (left), 128 chanel 3D neural
hypothesis. Squids use their brain to swim and to find their probe for ECoG by Neuro Nexus (right)
attachment place, after the attachment to the bottom of the
ocean, it digests its own brain. Thanks to neuroscience a lot of
the brain was reverse engineered. The 10/20 electrode II. CURRENT STATE OF BCI
placement for EEG offer satisfactory spatial coverage for most In the early days, BCI was mainly used in medical
of the applications, but for sophisticated applications like applications. Nowadays as the performances, the discreteness

978-1-5090-3748-3/16/$31.00 ©2016 IEEE

and the costs have seen a great improvement, BCI can be used
in almost every domain where human to machine interaction
can be imagined. According to BNCI Horizon 2020, the
author embraces the five application areas for BCIs.

A. BCI used to replace functions

BCI can offer a solution for patients with disabilities
caused by chronic diseases. In case of stroke many people loss
their motor functions or their ability to speak. The first attempt
to create a new communication path using BCI for patients
with severe motor disabilities was based on cursor Figure. 2. Ear-EEG [4]
movements. According to G. E. Fabiani et. al [7] the overall
In BCI four paradigms are used, P300 signals, motor imagery,
system performance depend on the number of features and the
Steady State Visual Evoked Potentials (SSVEP) and Slow
classification method. The use of linear discriminant analysis
Cortical Potentials. A.A Tores-Garcia et. al. have proposed a
(LDA) can offer optimal performances for linear separable
new paradigm for speech classification called imagined
data. Non-linear classifiers (like. statistical classifiers or
speech. [14].
neuronal networks) can offer better performances but the
number of features needs to be carefully chosen thanks to the
exponential relation between the number of weights and the B. BCI used to restore functions
number of features. One of the most popular applications for BCI is a potential method for stroke rehabilitation. Using
BCI is wheelchair control. EEG-based methods suffer from signals from the brain to control assistive devices like assistive
lack of accuracy and low signal to noise ratio. In order to robots or muscle stimulators. Earlier studies have
maximize the overall system performance, various methods demonstrated that ipsilateral areas to the moving limb are
were proposed to ensure low-cost function and safety.Velocity responsible for execution and planning. D.T Bundy et. al.
and direction control proportional with the concentration level demonstrated for the first time that contralesional hemispheric
and time was embraced by many authors. The main EEG signals can be used to control cursor movements in one
disadvantage is caused by the rapid exhaustion of the user. dimension. [3].
According to R. Satti [9] a shared control strategy can offer
better performances, where the computer can assist the user to C. BCI used to improve functions
ensure safety and to minimize the cost function. The cost Thanks to novelty wearable EEG devices which can be
function in terms of wheelchair application is defined as the used long time periods without intervention BCI applications
sum of concentration time ratio and mission time ratio. In gained place also in health improvement. D. Looney et. al.
order to create shared control systems the wheelchair need to proposed an Ear-EEG device for continuous brain monitoring
be equipped with environmental sensors (i.e infrared sensors, applications. The proposed device consisted of an ear plug
vision, etc) and trajectory control to minimize the number of (fig. 2) with embedded electrodes to record brain waves within
commands and collision avoidance. According to L. Merino the ear canal. [4]. A recent application proposed by B.
et. al. [16] BCIs based on Steady State Visual Envoked Mirkovic et. al. is a hearing device control using BCI. [19].
Potentials (SSVEP) does not require training as Motor Using continuous EEG speech envelope tracking can be made
Imagery based BCIs. In the related work six 3x3 cm flickering in order to detect the direction of attention to a natural speech.
stimulus was used with color and frequency changing, each The above study proves that continuous EEG offer lower
flickering square indicating a command. The SSVEP accuracy than scalp EEG due to the lower number of
frequency detection was made using Canonical Correlation electrodes, but both offer sufficient decoding accuracy.
Analysis (CCA). The proposed method offered robustness and Another recent BCI application is related to memory
good accuracy across multiple users. The main benefit of this improvement using wearable EEG headset. According to M.
method is the easiness to add more commands without the Avaneh, et al. parietal and occipital EEG signal amplitude
need for a training session. Using a webcam feedback with the measurements from 1-2s after the onset can be used to identify
proposed method to control a quadcopter is a novelty poorly or well-memorized words. [20]
application of BCI. The second most studied BCI application
after movement is related to speech synthesizing. According to
D. BCI used to enhance functions
J. S. Brumberg, the discrete classification of vowels and
consonants is very difficult. Motor imagery is suitable for One of the main limitation in BCI performance is due to
continuous control like mouse control in 2D space. The above the inability of the users to learn to provide sterling EEG
study presents a method to generate the 8 vowel sounds in patterns. A small percent of the users suffer from BCI
English using motor imagery to navigate through a map of deficiency, which means they can't learn how to use a BCI.
sounds defined by the outer boundary of the vowels. (i.e ih One possible solution is to create interactive co-learning
/hid, uh/ hood, /eh head). During the training participants were environment with several feedback paths (i.e visual,
instructed to imagine right-hand movement for /aa/ vowels, somatosensory, auditory), in this manner, the user engagement
left-hand movement for /uw/ vowels and both feet movement can be considerably increased and the user gains more detailed
for /iy/ sounds. To provide improved BCI performance both information about his neuronal activity. (i.e. what is wrong
audio and visual feedback was presented for the participants. with the provided pattern). This game like feedback methods
offer not just better EEG patterns but also enhance the user III. FUTURE OF BCI
capabilities. Thanks to the modern lifestyle mood disorder is BCI is a rapidly growing field of research joining together
one of the most common illness. Numerous studies have various fields. The bellow table depicts various applications
shown the relationship between music and mood. Affective and their status on the market.
Brain-Computer Interfaces (aBCI) are a possible solution for
Application Time to market
mood enhancement. According to I. Daly et. al. [18] aBCI Control of devices 5-10 yrs
music interface can be used to modulate emotions. The User state monitoring 3-5 yrs
proposed system is consisted of EEG and other physiological Evaluation 1-3 yrs
signal recorder, an affective state detector based on support Training and education 3-5 yrs
Gaming and entertainment Now
vector machines, a reasoning system to determine the best way Cognitive improvement 3-5 yrs
to change the user affective state to a target affective state and Safety and security 5-10 yrs
a piano music generator. The above-mentioned solution can
Table 1. BCI market overview
detect the current affective states with a mean accuracy of 0.54
and it has a potential in novelty music therapy applications. Controlling an artificial actuator using BCI is one of the
Affective state monitoring has opened new communication most complex applications. Recently developed methods
pathways also for artists. One of the most popular application decode cognitive signals to control artificial device, the main
is Eunoia by Lisa Park, which can translate emotions (i.e drawback is the caused mental fatigue by the need for
sadness, anger, hatred, desire, happiness) into acoustic continuous concentration. In the near future shared control
vibrations in real-time. Severall studies have demonstrated the systems will appear, cognitive signals will be used to decode
cognitive benefits of playing video games. Unity 3D is a the main characteristics of movements (e. g. direction, the
highly embraced open source game and virtual reality engine. intent of the movement) and the detailed characteristics will be
C. Kapeller et. al. [17] have released a screen-overlay control accomplished by the actuator itself. Recent studies have
demonstrated the remapping of the internal body
interface (SOCI) plugin for Unity which can be used together
representation of the brain due to the long-term usage of
with Oculus Rift VR glasses. The plugin uses popular BCI
artificial actuators which lead to a perceptual experience that
paradigms as P300 and c-VEP delivered by an openGL flash the artificial device became an extension of the user's body.
module. Synchronous BCIs like Code-modulated Visual According to Mariushi et. al [24] activation of right parietal
Evoked Potentials (c-VEP) require a stable refresh rate with cortex have been recorded during myoelectric prosthetic hand
good synchronization with an external trigger. The above movements. In future among various classical feedbacks (e. g.
plugin ensures the requirements by constantly running the visual, auditory, haptic) direct brain micro-stimulation (e.g.
Unity game at 60 Hz refresh rate and an internal UDP handler deep brain stimulation, transcranial magnetic stimulation,
in the SOCI player provide an accurate trigger. transcranial direct current stimulation, etc) patterns will be
used to enhance the user experience and to minimize the
E. BCI used as research tool
learning time. Currently various BCI hardware is available on
In the early days, BCIs has focused mainly on communication the market starting from gadget type BCI headset for gaming
an control. Novel hardware and machine learning methods applications to expensive research headsets and signal
unlocked new applications for BCIs among replace, restore, amplifiers. On the other hand, various type of software was
enhance and improve applications it can be used as a research developed for feature extractors, classifiers, etc by researchers
tool in various fields. (e.g neuroscience, marketing, etc). Some or by companies. The lack of industrial standard represents a
recent studies revealed the influence of soft drinks to brain- huge impediment to the future development of BCIs. In order
computer interface. According to J. Mundahl [22], caffeine to facilitate the communication between stakeholders and
decreases the alpha power at rest but increase attention. different disciplines, various international projects were
Caffeine has negligible effects on BCI performance because founded. (e. g. BNCI Horizon 2020). In the future, industrial
the two effects cancel out each other. Sugar based drinks standards will be developed as connectors for EEG caps,
decrease the BCI performance. The above observations result software application interfaces and datasets for benchmarking.
from EEG data recorded over left and right motor cortex IV. FUTURE WORK
positions (C3, C4) for six participants before and after caffein
or sugar based drink consumption. In each case, the resting Our future contributions will be based on a hardware
state was recorded followed by a left-right cursor control. platform using a system on a chip combined with a memristor
Another interest in BCI is subject to subject transferability crossbar memory. The proposed hardware offer several
with a minimal training session. According to H. Cho et. al. in advantages over classical approaches where personal
the case of motor imagery BCI [21] if the individual features computers or smartphones are used for feature extraction,
are similar to standard features the subject transfer rate is classification, user interface like lower power consumption,
comparable with individual accuracy. A possible solution for higher computational performances, brain-like behavior,
subject to subject transfer is interactive neurofeedback [23] wearable. According to Maheshwar Pd. Sah [25] both axons
where the user learn how to use the system or co-adaptive way and synapses can be emulated using memristors. The proposed
where both the user and the computer learn.The above method can offer the advantages of traditional algorithms
methods do not help just for subject to subject transferability using ARM core processor, field programable gate arrays, and
but also can contribute to BCI illiteracy problem solving. novelty neuromorphic devices.
 V. CONCLUSIONS [9] R. Satti, D. Coyle, G. Prasad, "Self-paced brain-controlled wheelchair
methodology with shared and automated assistive control",
Brain-computer interfaces have great potential in almost [10] J. R. Wolpaw, N. Birbaumer, D. J McFarland, G. Pfurtscheller and T.M
every application where human to machine interface can be Vaughan, "Brain-Computer Interfaces for communication and control",
imagined from entertainment to artificial prosthesis control. Clinical neurophsiology, col. 113, no. 6, 2002
Thanks to the rapidly growing computational power and [11] D. Coyle, "Neural network based auto association and time-series
novelty sensor development, nowadays everybody has access prediction for biosignal processing in brain-computer interfaces",
to medical graded gadget EEG caps which permit larger area Computational Intelligence Magazine, IEEE, vol. 4, no. 4, pp. 315-318,
2009
of applications, i.e games, mental state monitoring etc. On the
[12] M. A Lebedev and M. A.L. Nicolelis, "Brain-Machine Interfaces: past,
other hand, the lack of standardization and a well-established present and future", Elsevier Science Publishing Co., Trends in
ethics can slow down the research progress. Understanding Neurosciences, Vol. 29, No. 9, 2006
and solving BCI illiteracy is one of the biggest challenge. [13] D. J. Krusienski, M. G. Wentrup, F. Galan, D. Coyle, K. J. Miller, E.
Feedback quality improvement and co-adaptive systems are Forney and Ch. W. Andreson, "Critical issues in state-of-the-art brain-
possible solutions both for BCI illiteracy and better system computer interface signal processing", Journal of Neural Enigneering,
performances. The invention of memristive systems has Vol. 8, 2011
opened the possibility to create neuromorphic chips which act [14] A.A Torres-Garcia, E. F. Gonzalez-Castaneda, C.A. Reyes-Garcia, L. V.
Pineda, "On the pursuit of classification of EEG recorded during
as a biological neuron. Using a hybrid system using well- imagined speech", Proceeding of the 6th International Brain-Computer
known signal processing combined with hardware based Interface Meeting, 2016, in press
classifiers, the overall system performance can be improved [15] J. S. Brumberg, J.D. Burnison, K. Pitt, "Development of a real time
due to its adaptive behavior. speech synthesizer based brain computer interface", Proceeding of the
6th International Brain-Computer Interface Meeting, 2016, in press
[16] I. Merino, P. Kolar, T. Nayak, G. Hall. D. Pack, Y. Huang "Controlling
UAVs with SSVEP-based BCI" Proceeding of the 6th International
Brain-Computer Interface Meeting, 2016, in press
[17] C. Kapeller, N. Sabathiel, C. Hintermuller, B. Z. Allison, C. Guger,
"Unity plugin for immersive BCI applications" Proceeding of the 6th
International Brain-Computer Interface Meeting, 2016, in press
REFERENCES [18] I. Daly, D. Williams, A. Kirke, J. Weaver, A. Malik, F. Hwang, M.
Wairagkar, E.R. Miranda, S. J. Nasuto, "An affective Brain-Computer
music interface, " Proceeding of the 6th International Brain-Computer
[1] https://ec.europa.eu/programmes/horizon2020/en/news/bnci-horizon- Interface Meeting, 2016, in press
2020-retreat [19] B, Mirkovic, M, G, Bleichner, S. Debener, M. De vos, "Speech envelope
[2] C. Gruger and B. Z. Allison, "Recent advances in Brain-Computer tracking using around-the-ear EEG", Proceeding of the 6th International
Interface research- The BCI award 2013" SpringerBriefs in Electrical Brain-Computer Interface Meeting, 2016, in press
and Computer Engineering, 2014 [20] M. Avaneh, R. Mc Cormack, T. E. Ward, I.H. Robertson, "Improving
[3] D. T. Bundy and E. C. Leuthardt "An ipsilateral, contralesional BCI in memory performance using a wearable BCI" Proceeding of the 6th
chronic stroke patients" SpringerBriefs in Electrical and Computer International Brain-Computer Interface Meeting, 2016, in press
Engineering, 2014 [21] H. Cho, S. Ahn, S. Chan Jun "How is subject-to-subject transfer
[4] D. Looney, P. Kidmose, M. J. Morell and D. P. Mandic "Ear-EEG: probable in motor imageri BCI?", Proceeding of the 6th International
continuos brain monitoring" SpringerBriefs in Electrical and Computer Brain-Computer Interface Meeting, 2016, in press
Engineering, 2014 [22] J. Mundahl, J. Meng, J. He, B. He, "Soft drink effects on brain computer
[5] D. J. Krusienski, G. Schalk, D. J. McFarland, J.R Wolpaw, "Tracking of interface online performance and resting-state arousal", Proceeding of
the Mu rhythm using an empirically derived matched filter", 2nd the 6th International Brain-Computer Interface Meeting, 2016, in press
International IEEE EMBS Conference on Neural Engineering, 2014 [23] I. Kuzovkin, K. Tretyakov, A. Uusberg, R. Vicente, "Adaptive
[6] J.R Wolpaw, "Brain-computer interfaces as a new brain output interactive learning for training BCI systems" Proceeding of the 6th
pathways", Jurnal of Physiology, 2005 International Brain-Computer Interface Meeting, 2016, in press
[7] G. E. Fabiani, D. J. McFarald, J. R. Wolpaw, G. Pfurtscheller, [24] Mariushi M (2004) "Brains activation during manipulation of the
"Conversion of EEG activity into cursor movement by a Brain- myoelectric prosthetic hand: a functional magnetic reonance imaging
Computer Interace", IEEE Transactions on Neural Systems and study:" NeuroImage, 21
Rehabilitation Engineering, Vol. 12, No. 3, September, 2004 [25] Maheschwar Pd. Sah, Hyongsuk Kim, Leon. O. Chua "Brains are made
[8] P. Brunner, L. Bianchi, C. Guger, F. Cincotti and G. Schalk, "Current of memristors", IEEE Circuits and systems magazine, First quarter 2014,
trends in hardware and software for Brain-Computer Interfaces", Jurnal page 12-36.
of Neural Engineering, Vol. 8, 2011