A REAL-TIME FRACTAL-BASED BRAIN STATE

RECOGNITION FROM EEG AND ITS APPLICATIONS

Olga Sourina, Qiang Wang, Yisi Liu and Minh Khoa Nguyen
Nanyang Technological University, Nanyang Avenue, Singapore, Singapore

Keywords: BCI, Neurofeedback, Emotion recognition, Fractal dimension, EEG, Real-time applications.

Abstract: EEG-based immersion is a new direction in research and development on human computer interfaces. It has
attracted recently more attention from the research community and industry as wireless EEG reading
devices became easily available on the market. EEG-based technology has been applied in anaesthesiology,
psychology, serious games or even in marketing. As EEG signal is considered to have a fractal nature, we
proposed and developed a novel spatio-temporal fractal based approach to the brain state quantification. The
real-time algorithms of emotion recognition and concentration level recognition were implemented and
integrated in human-computer interfaces of EEG-enable applications. In this paper, EEG-based “serious”
games for concentration training and emotion-enable applications including emotion-based music therapy
on the Web were proposed and implemented.

1 INTRODUCTION activities by using suitable signal processing and

classification methods and algorithms.
Immersive human interaction with computer systems We proposed new algorithms of brain state
should use all human senses such as visual, audio, recognition including emotion recognition and
tactile, odour, taste, etc to make virtual or even non- concentration level recognition, and innovative
virtual experience more real. In the case of human integrated methods and tools for implementation of
computer systems, human receives information from the EEG-based user immersion and interaction.
the system using eyes, ears, skin, nose, etc and the Algorithms of the “inner” brain state quantification
information is processed by the corresponding lobes including emotion recognition would advance
of the brain. Then, the user enters information into research on human computer interaction bringing up
the computer system using the implemented human- the proposed novel objective quantification methods
computer interface system scenarios. To make the and algorithms as new research tools in medical
human-computer interfaces more seamlessness the applications, entertainment, and even novel digital
information could be entered involuntary as well. It art methodology applications, and allowing us an
could be entered into the computer by cameras, integration of the brain state quantification
sensors based tracking systems, and by biofeedback algorithms in the human computer interfaces. It
sensors. Then, the information could be processed would lead to the implementation of the applications
by the corresponding algorithms depending on the such as EEG-based serious games including
system application. In this paper, we study a novel neurofeedback games, emotion-enable personalized
dimension of human-computer interfaces that is search on the Web, experimental art animation,
based on real-time EEG recordings and its personalized avatars seamlessness communicating
recognition. Electroencephalogram (EEG) is a non- with virtual objects, other avatars, or even social
invasive technique recording the electrical potential robots working with elderly people, etc.
over the scalp which is produced by the activities of In this paper, we describe a novel spatio-
brain cortex, and reflects the state of the brain temporal fractal-based approach to brain state
(Nunez and Srinivasan, 2006). EEG technique gives recognition particularly to recognition of the
us an easy and portable way to monitor brain concentration levels and emotion recognition. The
method is general, and it could be used as a basis for
the development of other brain states recognition

82 Sourina O., Wang Q., Liu Y. and Nguyen M..

A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS.
DOI: 10.5220/0003335300820090
In Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011), pages 82-90
ISBN: 978-989-8425-35-5
Copyright c 2011 SCITEPRESS (Science and Technology Publications, Lda.)
 A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

algorithms as well. Our approach consists from two Related Potential (ERP) distortion can reflect
parts: interactive tools allowing dynamic analysis of psychological disorders such as Attention Deficit
EEG signals amplitudes and other parameters Hyperactivity Disorder (ADHD) (Lubar et al., 1995,
distribution over the brain lobes, and fractal Fuchs et al., 2003), Autistic Spectrum Disorders
dimension algorithms with sliding window that can (ASD) (Coben et al., 2010, Kouijzer et al., 2010),
estimate complexity of the signal by calculating Substance Use Disorders (SUD) including alcoholics
fractal dimensions values changing over time. The and drug abuse (Saxby and Peniston, 1995,
algorithms of concentration level recognition and Sokhadze et al., 2008), etc. Neurofeedback can be
emotion recognition use just one fractal feature per used for treating these disorders besides medical
channel that allows us to implement real-time EEG- treatments. Many neurofeedback games were
enable applications. We implemented real-time assessed, and it was proved that they have a healing
applications such as blobby 3D mapping, effect on patients with ADHD while the patient has
concentration games, emotion-enable search on the abnormal θ/β ratio of EEG. Besides the ratio, the
Web, etc. that are described in this paper. distortion in Slow Cortical Potential (SCP) was also
In Section 2.1, we describe Brain Computer notified in (Gevensleben et al., 2009). Both the
Interfaces (BCI) and neurofeedback techniques. In frequency band neurofeedback training and the SCP
Section 2.2, emotion recognition algorithms are neurofeedback training could achieve a good healing
reviewed. Then, we describe the proposed spatio- effect for ADHD (Gevensleben et al., 2009). Two
temporal fractal-based approach to the brain state EEG signal processing methods are prevalent in BCI
recognition. In Section 3.1, 3D Mapping of EEG systems: power spectrum analysis for different
signal with blobby model is presented. In Section frequency bands and event related potential analysis.
3.2, fractal dimension algorithms applied for As the different frequency band reflects different
extraction of fractal dimension features of EEG brain functions (Demos, 2005), frequency training is
signal with sliding window are elaborated. An a well-known technique applied in clinic
implementation of the overall algorithm is given in applications together with the Quantitative EEG
Section 4. Real-time EEG-enable applications (QEEG) protocol. In QEEG protocol, the power over
including EEG-based serious games and emotion- different bands is assessed from the patients EEG
based music search are described in Section 5. signals, and compared to the reference QEEG
database. Pathology and the corresponding recovery
protocol can be generated with the statistical model.
2 RELATED WORK The ERP analysis including SCP and P300 analysis
is a technique to analyze the event synchronized
EEG potential. SCP has shown its usability in
2.1 BCI and Neurofeedback ADHD treatment in (Gevensleben et al., 2009), and
P300 component training could be used for drug
Brain Computer Interfaces (BCIs) are the systems
abuse rehabilitation (Sokhadze et al., 2008).
that use brain signals to create a new channel of
Although an efficiency of EEG linear features
interaction of humans with computers or other
application were proved in clinical treatments, the
devices. Mostly the systems are used for disabled or
nonlinear methods, e.g. entropy analysis and fractal
elderly persons. Recently, efforts have been made on
dimension analysis, became popular in EEG
the development of EEG-based real-time
processing due to the nonlinearity of the EEG
applications in multimedia communication,
signals. The hypothesis is that a non-linear fractal
rehabilitation games, interaction in virtual
dimension approach allows quantify brain states
environments, etc. Traditionally, neurofeedback is a
corresponding to the concentration levels, pain
technique that allows the user voluntary change
levels, etc. In work (Wang et al., 2010b, Wang et al.,
his/her brain state based on the visual or audio
2010a), two well-known algorithms such as Box-
feedback from the system corresponding to the
counting (Block et al., 1990) and Higuchi (Higuchi,
recognized from the user EEG state of the brain. The
1988) were applied in concentration level
user by doing some exercises recommended by the
recognition in neurofeedback games, and the
doctor or just by playing the serious game with
efficiency was approved.
neurofeedback learns how to improve his/her brain
plasticity. Neurofeedback could recover some
psychological disorders or just help to improve some
skills of concentration, meditation, etc. Some
research demonstrates that both the EEG and Event

83
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

2.2 Emotion Recognition Algorithms Table 1: Off-line emotion recognition algorithms.

Feature
Emotion recognition from EEG could reveal the Author Emotion Result
and Classification
“inner” feeling of the user, and then, it could be used Subject-dependent emotion recognition works
in a therapy or to create an emotion-enable avatar of Joy:
Feature 54.5%
the user or other real-time applications. Emotion FFT; Joy, Anger:
(Ishino and
recognition algorithms consist from two parts: Wavelet transform; sad, 67.7%
Hagiwara,
feature extraction and classification. For real-time Variance, mean angry, Sorrow:
2003) 59%
applications, an objective is to develop fast Classification relaxed
Relaxati-on:
algorithms recognizing more emotions with fewer Neural Network
62.9%
electrodes used. Currently, mostly off-line Feature
recognition algorithms were proposed which are PCA Negative
(Zhang and
shown in the Table 1. EEG-based emotion Classification and 73.00%
Lee, 2009)
recognition algorithms could be divided into two Linear Kernel SVM; positive
groups: a subject-dependent and a subject- RBF Kernel SVM
independent one. In the Table 1, the algorithms are Feature
compared by feature extraction and classification 6 frequency bands from
algorithms used, by emotion types recognized and (Chanel et
different locations
3 degree of
by the algorithms accuracy. The algorithms are also Classification 58%
al., 2006) arousal
differed by the number of the electrodes used in the Naïve Bayes;
emotion recognition. In the Table 1, in works (Ishino Fisher Discriminant
and Hagiwara, 2003, Zhang and Lee, 2009, Analysis
Takahashi, 2004, Petrantonakis and Hadjileontiadis, Feature
Short Time Fourier Positive/
2010) 3 or 2 electrodes were used. All other works
Transform; Mutual arousal,
employed more than 32 electrodes to collect EEG
(Chanel et Information neutral/
data. 63%
al., 2009) Classification calm,
In (Liu et al., 2010), we proposed real-time
Discriminant Analysis; negative/
algorithm only using 3 channels in total. Fractal
SVM; Relevance Vector arousal
dimension algorithms were applied to compute Machine
fractal based features, and a real time EEG-based Feature: Joy,
emotion recognition algorithm was implemented (Lin et al., ASM 12 anger,
with predefined thresholds based on the training 90.72%
2009) Classification sadness,
session analysis. In our work, by recognizing arousal SVM pleasure
and valence level with an accuracy of 84.9% and Subject-independent emotion recognition works
90% respectively, the satisfied, pleasant, happy, Feature
frustrated, sad, fear, and neutral emotions were Statistical feature Calm,
differentiated. Since the discrete emotions can be (Khalili and combined with positive
mapped to the 2D emotion model, and fractal Moradi, Correlation dimension aroused, 76.66%
dimension values can be mapped to 2D emotion 2009) Classification negative
model as well, more emotions that are defined in 2D Quadratic Discriminant aroused
model could be distinguished. Analysis
Joy, anger, 41.7% for
Feature five
sadness,
(Takahashi, statistical features emotions,
fear and
3 SPATIO-TEMPORAL 2004) Classification
realizati-
66.7% for
three
SVM, Neural Networks
APPROACH on emotions
Feature: 62.3% for
Happy,
The spatio-temporal approach combines two (Petrantona
Statistical features,
surprised,
single
channel
methods: a spatio-temporal analysis and fractal kis and
wavelet based features,
angry, case,
based analysis. The spatio-temporal analysis higher order crossings. 83.33% for
Hadjileontia fear,
includes real-time 3D mapping of EEG signal Classification: combined
dis, 2010) disgust,
SVM, QDA, KNN, channel
amplitude or other parameters, for example, fractal sad case
Mahalanobis Distance
dimension values, with blobby model defined by
implicit functions and applying set-theoretic

84
 A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

operations over the moving shapes to isolate of all shapes to show constant activity on the time
activities common for the signal during the time interval, and union of all shapes to show the overall
interval, as well as those that are unique one. The maximum activity. On two data sets, we could apply
proposed fractal based method allows us to estimate an intersection to show common activity, union to
the signal complexity changing over time and then, show overall maximum activity and subtraction to
recognize the brain state. show activities which are characteristic to one set.

3.1 3D Mapping of EEG 3.2 Fractal-based Approach

We proposed a novel method of EEG analysis based Fractal dimension (FD) is a measurement of
on 3D mapping of EEG data. We employed a complexity and irregularity of the object based on an
concept of a dynamic 3D volumetric “blobby” shape entropy analysis. Entropy is a measure of the
to visualize the EEG signal changes over time. The disorder in physical systems, or an amount of
blobby-like objects were firstly introduced in (Blinn, information that may be gained by observations of
1982, Wyvill et al., 1986). A time-dependent the disordered systems. A common practice to
“blobby” object is defined using implicit functions distinguish among possible classes of time series is
that allow us to propose and implement set-theoretic to determine their so-called correlation dimension.
operations over the time changing shapes for further The correlation dimension, however, belongs to an
analysis. infinite family of fractal dimensions (Hentschel and
This object is defined using so-called FRep Procaccia, 1983). Hence, there is a hope that the use
representation proposed in (Pasko et al., 1995) and of the whole family of fractal dimensions may be
extended to spatio-temporal model in (Kulish et al., advantageous in comparison to using only some of
2006a, Sourina et al., 2009) where it was described these dimensions. The concept of generalized
by the following formula: entropy of a probability distribution was introduced
M
by Alfred Renyi (Renyi, 1955). Based on the
f ( x, y , z , t ) = ∑a e
i =1
i
− ri .bi ( t )
−g ≥0
(1)
moments of order q of the probability pi , Renyi
obtained the following expression for entropy
ri = ( x − xi ) 2
+ ( y − yi ) + ( z − zi )
2 2
1
N
Sq =
q −1
log ∑p q
i (2)
where a is a scale factor, b is an exponent scale i =1
factor changing over time, g is a threshold value, and where q is not necessarily an integer and log denotes
m is the number of electrodes used. log2. Note that for q → 1 , Eq. (2) yields the well-
At any given point (x, y, z, t), function f can take
known entropy of a discrete probability distribution
negative, positive or zero values. The point is
(Shannon, 1998):
considered on the surface of the object if the
N
function value is zero, inside the object if the
function value is positive, and outside the object S1 = − ∑ p log p
i =1
i i (3)
otherwise. The shape changes through time due to
the variable values of the exponent factor b There are various methods to calculate fractal
according to the signal. Its size and appearance dimensions. In works (Kulish et al., 2006b, Kulish et
visually reflect the brain activity. For a better visual al., 2006a), the generalized Renyi approach based on
impression, the blobby shape is superimposed on a Renyi entropy and calculation of the whole spectra
3D head model. Besides just a visual comparison, of fractal dimensions to quantify brain states were
we proposed to apply set-theoretic (“Boolean”) studied. In our real-time applications, we apply only
operations to the moving shapes to isolate activities Hausdorff dimension when q = 0 in (2). We
common for both of them per time point, as well as implemented two well-known Higuchi (Higuchi,
those that are unique for either one. Furthermore, the 1988) and Box-counting (Block et al., 1990)
group set-theoretic operations applied to the algorithms calculating fractal dimension. Both of
individual time frames of the moving shape allow us them were evaluated using mono-fractal Brownian
to isolate idle parts of the brain as well as to estimate and Weierstrass functions where theoretical FD
an average level of the brain activity. The proposed values are known (Wang et al., 2010a). Higuchi
operations could be applied over one or/and over algorithm gave a better accuracy as FD values were
two datasets. On one data set, we can do intersection closer to the theoretical FD ones.

85
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

The algorithms were used in the proposed X 31 : x (1) , x ( 4 ) , x ( 7 ) ,..., x (100 )

algorithm of FD feature extraction with sliding
window for concentration level recognition and X 32 : x ( 2 ) , x ( 5 ) , x ( 8 ) ,..., x ( 98 )
emotion level recognition.
The Box-counting and Higuchi algorithms are X 33 : x ( 3) , x ( 6 ) , x ( 9 ) ,..., x ( 99 )
described as follows.
Then, every length of each sub-series X km is
3.2.1 Box-counting Method
calculated. Length Lm (k ) of X km is equal to
Fractal dimension DB is defined in Box counting
⎧ ⎛ N −m
⎞ ⎫
method (Block et al., 1990) as ⎪ ⎜
⎪ ⎝
∑ k
i=1
x ( m + i k ) − x ( m + ( i − 1 ) .k ) ⎟
⎠
(N − 1)⎪
⎪
⎨ ⎬
⎡ N − m ⎤ (7)
ln N ( ε ) ⎪
⎩⎪
⎢⎣ k ⎥⎦ . k ⎪
⎭⎪
DB  − lim
ε →0 ⎛1⎞ (4) k
ln ⎜ ⎟
⎝ε ⎠ Step 3: Calculate the average length L(k ) of all
Equivalently, Lm (k ) .
ln N ( ε ) Step 4: Repeat step 1 to 3 for several values of
DB = − lim (5) k.
ε →0 ln ( ε )
Step5: Slope of the curve of ln( L(k )) versus
where N (ε ) is the number of boxes of length ε ln(k ) is approximated. Fractal dimension value is
which cover the whole data set. the slope multiplied by -1.
Our implementation of the above box counting
formula is based on (Phothisonothai and Nakagawa,
2007). A time series data are covered by a grid of 4 IMPLEMENTATION
boxes of length ε . N (ε ) is the number of boxes
that the curve intersects with the grid. Different The proposed real-time system diagram is shown in
values of ε result in different number of boxes and Figure 1. The user receives stimuli from the
hence different values of N (ε ) . Therefore, the curve computer system such as visual, audio, etc. Then,
of ln N (ε ) is plotted versus ln(ε ) and DB is the mental process of the user thinking is recognized
from his/her EEG that is acquired by the EEG
calculated as the slope of that curve multiplied by device. An overall recognition algorithm used in the
−1 . real-time applications consists from the following
steps: data sampling and pre-processing including
3.2.2 Higuchi Method
data filtering, feature extraction, and subject-
The following implementation is based on work dependent machine learning algorithm. Then the
(Higuchi, 1988). command to the feedback system is formed based on
Suppose we want to calculate fractal dimension the recognition results.
for a time series x (1) , x ( 2 ) ,..., x ( n )
Step 1: Choose one value of k
Step 2: Construct the sub-series X km from the
time series as following

⎛ ⎡ N −m⎤ ⎞
x ( m ) , x ( m + k ) ,..., x ⎜ m + ⎢ ⎥ .k ⎟ (6)
⎝ ⎣ k ⎦ ⎠

where m = 1, 2,..., k and [ ] denotes Gaussian

notation which rounds a number in the brackets to its
largest integer which is equal to or smaller than
itself, m the initial time and k the time interval. For Figure 1: Diagram for non-invasive BCI system. ^
example, when k = 3 and n = 100 we have 3 sub-
series as follows:

86
 A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

4.1 Pre-processing 5.1 3D Mapping of EEG

The collected data are filtered by a 2-42 Hz We proposed a spatio-temporal approach to EEG
bandpass filter since major waves of EEG lie in this analysis. 3D blobby-based EEG mapping was
band (Sanei and Chambers, 2007). implemented for offline processing. To monitor and
analyze the subject/user brain state in real time, a
4.2 Features Extraction system “VisBrain” was implemented. Signal
amplitude values are visualized with blobby model,
The next step after the data pre-processing is feature color or “pins”. In Figure 2, spatio-temporal
extraction. We apply a sliding window and calculate visualization of EEG signals is shown with the 3D
one FD value per sample per channel. Number of blobby mapping. The blobby model allows assessing
channels used in the recognition algorithm defines a a spatio-temporal pattern of the subject/user EEGs
size of the feature vector as follows: corresponding to different brain states.
F = {FD1 , FD2 ,...FDm } (8)
where m is number of channels
In the concentration level recognition algorithm,
we have one feature as only one channel is used
(Wang et al., 2010a). In the emotion recognition
algorithm, there are 3 features in the vector as 3
channels are used (Liu et al., 2010). Thus, in our
approach we use one FD value per channel.
Figure 2: Real-time system “VisBrain” for spatio-temporal
4.3 Classification Algorithms analysis.

Currently, we implemented a simple real-time 5.2 EEG-based Serious Games

subject-dependent classification algorithm based on
threshold FD values that are calculated during a The EEG-based serious game design includes two
short training session. Note that off-line processing parts: signal processing algorithms and a 2D/3D or
with SVM classifier of the EEG labelled with virtual reality game part. Raw EEG signals collected
emotions and concentration levels gave us similar by the device from the user brain are filtered and
accuracy as the real time implementation algorithm analyzed by signal processing algorithms, and the
used thresholds. resulting values are interpreted in the game as an
additional game control using just the “brain
power”. A therapeutic effect of such games consists
from combination of a distraction effect of the game
5 APPLICATIONS and an effect from the learning by the user/patient
how to control the game by changing voluntary
EEG data are collected by Emotiv device with 14 his/her brain state, for example, learning how to
electrodes located at AF3, F7, F3, FC5, T7, P7, O1, improve the user’s concentration level. We
O2, P8, T8, FC6, F4, F8, AF4 standardized by the developed two concentration games named “Brain
American Electroencephalographic Society (1991). Chi” and “Dancing Robot”, and one game for stress
The sampling rate is 128Hz. To be able to use any management named “Pipe”. They are simple single-
EEG device a program reading raw EEG signals is player games implemented with the game engines
needed to be implemented. Currently, our SDL, Panda3D, and Adobe Flash CS4
applications could also work with Pet 2 and Mindset correspondingly. The recognized relaxation/
24. All electrodes can be active in the system. The concentration/ stress level values from EEG could be
steps of an overall algorithm of the real-time interpreted in the games as any visual/audio effects
application are as follows. First, raw data are read or even as a behaviour change of the game
from the EEG device, filtered with band pass filter characters.
2-42 Hz, and entered to the corresponding brain state In the “Brain Chi” game, the relaxation/
recognition algorithm. Then, the results of the concentration level of the user is associated with
recognition are fed to the developed game, web site, radius of a “growing/shrinking” ball. It allows the
or any other real-time software. “little boy” character to fight enemies by “growing”

87
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

the ball. In the “Dancing Robot” game, the 2D Valence-Arousal emotion model. It is possible to
relaxation/concentration level is associated with the recognize in real time any discrete emotions that
“robot” character behaviour. When the concentration could be defined with the 2-dimensional emotion
level of the user increases, the robot character starts model. Satisfied, pleasant, happy, frustrated, sad,
to move faster. If the user is fully relaxed, the robot fear, and neutral emotions were recognized. In our
stops dancing. In our implementation, the algorithm, only 3 channels are used. We
concentration and relaxation levels could be easily implemented two emotion-enable real-time
associated either with concentration training or applications. First, we implemented an application
relaxation training depending on the therapeutic with the EEG-enable avatar (Liu et al., 2010). The
purpose of the game. In Figure 3, a change of the music stimuli were used for emotion induction as it
quantified level of the user concentration level is was proposed in (Sourina et al., 2008). We used an
interpreted as a “faster/slower” movement of the avatar available with free version of Haptek
“robot”. development package for our application (Haptek).

Figure 3: EEG-enable concentration “Dancing Robot”

game.
(a)

The “Pipe” game is implemented more as a

traditional neurofeedback game. In Figure 4, the
“blue” bar located under the “green” bar on the
screen shows the level of the user stress. In the
“Pipe” game, water flows faster when the player’s
stress level increases, and hence it makes the game
playing more difficult.
We also did preliminary study how to use the
EEG-enable serious games for pain management and
have got some promising results.
(b)

Figure 5: Emotion-enabled applications: a) “Pleasant”

emotion is recognized and visualized on the user 3D
avatar; b) “Sad” emotion is recognized and sent to the
music therapy Web site.

Haptek Activex control provides functions and

commands to change facial expressions of 3D
avatars. We defined six emotions by changing the
parameters controlling the facial muscles of the
Haptek emotion avatar. Those emotions are: fear,
Figure 4: EEG-enable stress management “Pipe” game. frustrated, sad, happy, pleasant and satisfied. In the
application, emotions of the user are recognized
5.3 Emotion-based Digital Experience from EEG and visualized in real-time on the user’s
avatar with Haptek system. In Figure 5(a), the user
We proposed and implemented a real-time fractal- was listening to music pieces for emotion induction,
based emotion recognition algorithm where the and the algorithm recognized “pleasant” emotion
calculated fractal dimension values were mapped to that was visualized on the user’s avatar. The avatar

88
 A REAL-TIME FRACTAL-BASED BRAIN STATE RECOGNITION FROM EEG AND ITS APPLICATIONS

emotions are changed according to emotions that the REFERENCES

user is feeling during the music listening. Second,
we implemented an EEG-enable music therapy web- 1991. American electroencephalographic society
site. The user/patient’s emotion is recognized from guidelines for standard electrode position
EEG, and the corresponding music piece is nomenclature. Journal of Clinical Neurophysiology, 8,
downloaded according to the emotion recognized 200-202.
from the EEG of the user. In Figure 5(b), emotions Blinn, J. F. 1982. Generalization Of Algebraic Surface
are induced by music stimuli played through the Drawing. Computer Graphics (ACM), 16, 273.
earphone, and then the “inner” user’s emotion is Block, A., Von Bloh, W. & Schellnhuber, H. J. 1990.
Efficient box-counting determination of generalized
recognized from the EEG signal in real-time. Then,
fractal dimensions. Physical Review A, 42, 1869-1874.
a pleasant song is played to the user upon Chanel, G., Kierkels, J. J. M., Soleymani, M. & Pun, T.
recognizing the user is being sad to improve his/her 2009. Short-term emotion assessment in a recall
mood or the corresponding music is played to calm paradigm. International Journal of Human Computer
down the user if he/she is too excited (happy or Studies, 67, 607-627.
angry) or too nervous feeling “fear” emotion. A Chanel, G., Kronegg, J., Grandjean, D. & Pun, T. 2006.
simple music therapy algorithm was proposed Emotion assessment: Arousal evaluation using EEG's
allowing automatically down load the corresponding and peripheral physiological signals
music piece based on the recognized emotion to Coben, r., Linden, M. & Myers, T. E. 2010.
Neurofeedback for autistic spectrum disorder: A
change the user mood. The choice of the appropriate
review of the literature. Applied Psychophysiology
music from the list could be given the user as well. Biofeedback, 35, 83-105.
Demos, J. N. 2005. Getting Started with Neurofeedback,
New York, WW Norton & Company.
6 CONCLUSIONS Fuchs, T., Birbaumer, N., Lutzenberger, W., Gruzelier, J.
H. & Kaiser, J. 2003. Neurofeedback treatment for
attention-deficit/hyperactivity disorder in children: A
In this paper, we proposed and described a novel comparison with methylphenidate. Applied
spatio-temporal fractal based approach to study Psychophysiology Biofeedback, 28, 1-12.
different brain states such as concentration levels, Gevensleben, H., Holl, B., Albrecht, B., Schlamp, D.,
human emotions, and in future other brain states Kratz, O., Studer, P., Wangler, S., Rothenberger, A.,
such as “central pain” feeling, attention levels, etc. Moll, G. H. & Heinrich, H. 2009. Distinct EEG effects
Using just one fractal dimension feature per channel related to neurofeedback training in children with
and the simple machine learning algorithm allows us ADHD: A randomized controlled trial. International
to implement real-time brain recognition Journal of Psychophysiology, 74, 149-157.
HAPTEK. Available: http://www.haptek.com [Accessed].
applications with acceptable accuracy that could be hentschel, H. G. E. & Procaccia, I. 1983. The infinite
improved by subject-based training. We expect number of generalized dimensions of fractals and
further use of the proposed approach in different strange attractors. Physica D: Nonlinear Phenomena,
real-time applications. It could be also used in 8, 435-444.
studies such as validation of the hypotheses: emotion Higuchi, T. 1988. Approach to an irregular time series on
induction could change pain level in the patients; the the basis of the fractal theory. Physica D: Nonlinear
positive emotions could improve human Phenomena, 31, 277-283.
performance, etc. We also work on the improvement IDM-PROJECT. 2008. Emotion-based personalized
on the real-time filtering of artefacts of different digital media experience in Co-Spaces [Online].
Available: http://www3.ntu.edu.sg/home/eosourina/
origin. The work described in the paper is a part of CHCILab/projects.html [Accessed].
the project EmoDEx presented in (IDM-Project, Ishino, K. & Hagiwara, M. Year. A feeling estimation
2008). system using a simple electroencephalograph. In,
2003. 4204-4209.
Khalili, Z. & Moradi, M. H. Year. Emotion recognition
system using brain and peripheral signals: Using
ACKNOWLEDGEMENTS correlation dimension to improve the results of EEG.
In: Proceedings of the International Joint Conference
This project is supported by the grant NRF2008 on Neural Networks, 2009. 1571-1575.
IDM-IDM004-020 “Emotion-based personalized Kouijzer, M. E. J., Van Schie, H. T., De Moor, J. M. H.,
digital media experience in Co-Spaces” of National Gerrits, B. J. L. & Buitelaar, J. K. 2010.
Research Fund of Singapore. Neurofeedback treatment in autism. Preliminary
findings in behavioral, cognitive, and

89
BIOSIGNALS 2011 - International Conference on Bio-inspired Systems and Signal Processing

neurophysiological functioning. Research in Autism Stimuli. In: Proc of 13th International Conference on
Spectrum Disorders, 4, 386-399. Biomedical Engineering ICBME 2008, 3-6 December
Kulish, V., Sourin, A. & Sourina, O. 2006a. Analysis and 2008. 411-414.
visualization of human electroencephalograms seen as Sourina, O., Sourin, A. & Kulish, V. Year. EEG data
fractal time series. Journal of Mechanics in Medicine driven animation and its application. In: Proc of
and Biology, World Scientific, 26(2), 175-188. International Conference Mirage 2009, 4-6 May 2009
Kulish, V., Sourin, A. & Sourina, O. 2006b. Human Rocquencourt. 380-388.
electroencephalograms seen as fractal time series: Takahashi, K. Year. Remarks on emotion recognition from
Mathematical analysis and visualization. Computers in multi-modal bio-potential signals. In, 2004. 1138-
Biology and Medicine, 36, 291-302. 1143.
Lin, Y. P., Wang, C. H., Wu, T. L., Jeng, S. K. & Chen, J. Wang, Q., Sourina, O. & Nguyen, M. K. Year. EEG-based
H. Year. EEG-based emotion recognition in music "Serious" Games Design for Medical Applications. In:
listening: A comparison of schemes for multiclass In Proc. 2010 Int. Conf. on Cyberworlds, 2010a
support vector machine. In: ICASSP, IEEE Singapore. 270-276.
International Conference on Acoustics, Speech and Wang, Q., Sourina, O. & Nguyen, M. K. Year. Fractal
Signal Processing - Proceedings, 2009 Taipei. 489- Dimension Based Algorithm for Neurofeedback
492. Games. In: Proc. CGI 2010. SP25, 2010b Singapore.
Liu, Y., Sourina, O. & Nguyen, M. K. Year. Real-time Wyvill, G., Mcpheeters, C. & Wyvill, B. 1986. Data
EEG-based Human Emotion Recognition and structure for soft objects. The Visual Computer, 2,
Visualization In: Proc. 2010 Int. Conf. on 227-234.
Cyberworlds, 20-22 Oct, 2010 2010 Singapore. 262- Zhang, Q. & Lee, M. 2009. Analysis of positive and
269. negative emotions in natural scene using brain activity
Lubar, J. F., Swartwood, M. O., Swartwood, J. N. & O' and GIST. Neurocomputing, 72, 1302-1306.
Donnell, P. H. 1995. Evaluation of the effectiveness of
EEG neurofeedback training for ADHD in a clinical
setting as measured by changes in T.O.V.A. scores,
behavioral ratings, and WISC-R performance.
Biofeedback and Self-Regulation, 20, 83-99.
Nunez, P. L. & Srinivasan, R. 2006. Electric Fields of the
Brain, Oxford University Press.
Pasko, A., Adzhiev, V., Sourin, A. & Savchenko, V. 1995.
Function representation in geometric modeling:
concepts, implementation and applications. The Visual
Computer, 11, 429-446.
Petrantonakis, P. C. & Hadjileontiadis, L. J. 2010.
Emotion recognition from EEG using higher order
crossings. IEEE Transactions on Information
Technology in Biomedicine, 14, 186-197.
Phothisonothai, M. & Nakagawa, M. 2007. Fractal-based
EEG data analysis of body parts movement imagery
tasks. Journal of Physiological Sciences, 57, 217-226.
Renyi, A. 1955. On a new axiomatic theory of probability.
Acta Mathematica Academiae Scientiarum
Hungaricae, 6, 285-335.
Sanei, S. & Chambers, J. 2007. EEG signal processing,
Chichester, England ; Hoboken, NJ, John Wiley &
Sons.
Saxby, E. & Peniston, E. G. 1995. Alpha-theta brainwave
neurofeedback training: An effective treatment for
male and female alcoholics with depressive symptoms.
Journal of Clinical Psychology, 51, 685-693.
Shannon, C. E. 1998. The mathematical theory of
communication University of Illinois Press.
Sokhadze, T. M., Cannon, R. L. & Trudeau, D. L. 2008.
EEG biofeedback as a treatment for substance use
disorders: Review, rating of efficacy, and
recommendations for further research. Applied
Psychophysiology Biofeedback, 33, 1-28.
Sourina, O., Kulish, V. V. & Sourin, A. Year. Novel Tools
for Quantification of Brain Responses to Music

90