2017 International Conference on Intelligent Computing,Instrumentation and Control Technologies (ICICICT)

EEG based Emotion Recognition using SVM and

PSO
Nivedha R, Brinda M, Devika Vasanth, Anvitha M and Suma K.V
Dept. of Electronics & Communication
Ramaiah Institute of Technology
Bangalore, India

Abstract— Machine learning has fueled real breakthroughs in attention deficit hyperactivity disorder (ADHD) and anxiety
affective computing in making the machines more emphatic to disorders.
the user. This emotion recognition capability of machines enables In our paper, we classify the EEG signals into four
them to act according to the observed mental state. Human discrete emotional states namely, “happy”, “sad”, “angry” and
feelings and emotions are triggered by stimuli which are external
“relaxed” using Russell’s Circumplex model. Spectral features
or internal and manifest themselves in the form of pulse rate,
tone, facial expressions and many more. In this paper we classify like power spectral density and coherence and wavelet
human emotions using EEG signals into four discrete states, features including energy and entropy are extracted and fed to
namely happy, sad, angry and relaxed. The preprocessed signals an SVM classifier. We have also tried reducing the number of
from the DEAP database is used and spectral and statistical electrodes such that there is no substantial decrease in the
features are extracted by discrete wavelet transform. These accuracy. The paper can be summarized as Section II deals
features are classified using a SVM classifier and the with Literature Survey, Section III dealing with Dataset
performance of the classifier is optimized using the PSO Description, Section IV with Feature Extraction and
algorithm. An overall emotional accuracy of 80.625% was Classification, Section V explains the Methodology, Section
obtained for a combination of 32 electrodes with a valence and
VI contains Results and Discussion, Section VII deals with the
arousal accuracy of 86.25% and 88.125%.
Conclusion.
Keywords— EEG, Wavelet transform, Emotion recognition,
SVM, PSO.
II. LITERATURE SURVEY
Emotions are known to have a prominent role in analyzing
I. INTRODUCTION the state of mind of a person and in the interaction and
Emotions play a key role in understanding human behavior. communication among people. In recent times recognition and
They are believed to be extremely potential for analyzing the classification of human emotions from Electroencephalogram
condition of the human mind and hence this area of (EEG) has led to the development of brain computer interfaces
recognizing emotions is of interest to both psychologists and which empowers computers in understanding human emotions.
engineers. Today different forms of human-centric and According to Plutchik [1], there are eight basic states of
emotion as acceptance, anger, anticipation, disgust, fear, joy,
human-driven interactions with digital media have
sadness and surprise. Rest of the emotional states can be
revolutionized entertainment, cyber worlds, learning, neuro-
modeled using the basic states such as sadness and surprise
cognitive sciences and many more areas. The demand for make disappointment. Garrett et al [2] in their paper compared
computer applications to detect the current emotional state of the performance of linear and non linear classifiers for emotion
the user is ever growing. Emotion recognition is popularly classification. The authors observed that the nonlinear
done using text, speech, facial expressions or gestures. But classifiers produce better classification results. They obtained
emotions are not always what they are displayed as. There is and an average classification accuracy of 66% using Linear
an explicit separation between the physiological arousal, the Discriminant Analysis (LDA), 69.4% using Neural Networks
behavioral expression (affect), and the conscious expression of (NN) and 72% using Support Vector Machine (SVM).
an emotion (feeling). Automatic emotion recognition using Soleymani et al [3] in their paper used 32 channel electrodes to
EEG signals is now the most preferred technique. This is classify emotions based on valence and arousal values in
because facial expressions and gestures being a behavioral response to video stimuli. The authors’ calculated Power
expression can always be controlled by voluntary actions spectral density (PSD) from different bands using fast Fourier
while EEG signals concentrate on the inner emotions of a transform (FFT) and Welch algorithm and an SVM classifier
person. Emotional markers are present in EEG signals and it is with RBF kernel was employed to classify the samples using
a relatively easy and cheap method to measure the brain features from different modalities. They obtained a best
activity. In the current century scientists are diligently trying classification accuracy of 68.5 % for valence and 76.4 %for
to make computer interaction more natural. This can be arousal labels. Murugappan [4] in his paper used 2 sets of EEG
applied in designing wearable gadgets to detect real time channels (64 and 32) to classify emotions. The author extracted
a set of linear (power, standard deviation, and variance) and
human emotions, it helps psychiatrists in treating
non-linear (entropy) features using multi-resolution analysis of
psychological disorders like autism spectrum disorders (ASD), Wavelet Transform (WT). Audio-visual stimuli (film/video
clips) was used for inducing the discrete emotions and

978-1-5090-6106-8/17/$31.00 ©2017 IEEE 1597

Authorized licensed use limited to: M S RAMAIAH INSTITUTE OF TECHNOLOGY. Downloaded on December 29,2020 at 05:52:14 UTC from IEEE Xplore. Restrictions apply.
 2017 International Conference on Intelligent Computing,Instrumentation and Control Technologies (ICICICT

classification was done using KNN classifier. The author The subjective rating of the participants is mapped onto this
obtained a maximum classification accuracy of 82.87 % on 62 model and the emotion can be inferred as:
channels and 78.57% on 24 channels, respectively. According
to Chung et al. [5] for many pattern classification problems, a Pleasant + Activation = Happy
higher number of features used do not necessarily translate into Pleasant + Deactivation = Relaxed
higher recognition rate. The authors suggested Particle Swarm
Optimization (PSO) as a good feature selection algorithm and Unpleasant + Activation = Angry
support vector machines (SVMs) with the one-versus-rest Unpleasant + Deactivation = Sad
method as a fitness function of PSO for the classification
problem. They obtained a higher accuracy using PSO-SVM IV. FEATURE EXTRACTION AND SELECTION TECHNIQUES
and this means that not all features are needed to achieve total The Features extracted are Power Spectral Density, Magnitude
classification accuracy. M. Sreeshakthy et al. [6] have used the Squared Coherence Estimate, Energy and Entropy. Particle
EEG signals from the DEAP database. DWT is applied on the Swarm optimization is used for feature selection.
preprocessed signals and statistical and energy based features
are extracted from the five sub bands. These features were used A. Power Spectral Density(PSD)
to train a neural network and the alpha and gamma bands were The frequency response of a random periodic signal, called
selected based on the mean squared error. Cuckoo search and as Power Spectral Density, denotes the average distribution of
PSO algorithms are used to optimize the multilayer perceptron power as a function of frequency. In order to characterize the
network. They obtained an accuracy of 96.3% using PSO variations in an EEG signal it is important to analyze the
optimized neural network and the highest accuracy of 97.01% variations in the peak amplitude and frequencies. Hence EEG
was obtained on using cuckoo search with neural networks, spectral analysis comes to the fore front. Time to compute PSD
also this classification method was found to have the least features is relatively lesser and it has also been proposed by
mean squared error. many researchers that using PSD features enhances the
performance of the classifier [9] [10].
III. DATASET DESCRIPTION
ഘ
ିଶగቀ ቁ௡
A multimodal dataset called the DEAP dataset [7] for ܵሺ߱ሻ ൌ σ௅ିଵ
௡ୀ଴ ‫ݔ‬ሺ݊ሻ‫ݓ‬ሺ݊ሻ݁
ഘೞ (1)
analyzing the human affective states is used. It consists of EEG
signals collected from 32 healthy participants, aged between 19 Where ܵሺ߱ሻ is the windowed DFT.
and 32, with an equal male to female ratio. Music videos were
used to elicit emotions and the signals were recorded using a B. Magnitude Squared Coherence Estimate(MSCE)
32 channel BioSemi acquisition system at a sampling rate of Magnitude Squared Coherence, commonly known as
512 Hz. 32 Ag/AgCl electrodes were arranged according to the coherence, measures the interdependence of two signals
10-20 international system. Each participant was shown 40 reflecting the distribution across frequency of activity common
music videos of one minute duration each and subjectively to both the signals. Coherence features are likely to be of a
rated their degree of valence and arousal on a scale of 1-9. The greater benefit when the recorded signals have a non-zero
acquired signals were found to be distorted by eye blinking and phase synchrony [11]. The functional interactions across the
muscular movements and thus were preprocessed. The signals various regions of the brain can be well studied by using the
were down sampled to 128Hz and a band pass filter of 4-45 Hz coherence estimates.
was applied. We used this segmented data free from all the
ȁ௉ೣ೤ሺ௙ሻȁమ
artifacts. ‫ܥ‬௫௬ ሺ݂ሻ ൌ (2)
௉ೣೣሺ௙ሻ௉೤೤ሺ௙ሻ
A. Russell’s Circumplex model
We adopt Russell's Circumplex model [8] to represent the Where ‫ܥ‬௫௬ is MSCE of input signals x and y using Welch’s
emotional space. It represents the emotion on a two averaged periodogram method, ܲ௫௬ is the cross power spectral
dimensional plane where one dimension indicates "arousing density, ܲ௫௫ and ܲ௬௬ are the power spectral densities of x and y
and sleepy" and the other is "pleasant and unpleasant". Discrete
respectively.
emotional states such as 'happy', 'sad', 'angry', 'relaxed' can be
inferred from the degree of valence and arousal as shown in the C. Wavelet Sub-Band Energy and Entropy
Fig. 1. EEG signal contains several spectral components. The
amplitude lies in the range of 10 to 100 microvolt and the
important frequencies are in the range of 0.1 to 30 Hz. The
standard EEG subbands are delta (0.1 to 3.5 Hz), theta (4 to
7.5 Hz), alpha (8 to 13 Hz), beta (14 to 30 Hz) and gamma
(greater than 30Hz) bands. These bands contain information
pertaining to brain activity which can be extracted using
Discrete Wavelet Transform (DWT). DWT presents a signal
as a combination of scaling functions and their wavelets at
different locations (positions) and scales (duration).DWT
decomposes a given signal into approximate and detailed
Fig. 1. Russell’s Circumplex model of emotions

978-1-5090-6106-8/17/$31.00 ©2017 IEEE 1598

Authorized licensed use limited to: M S RAMAIAH INSTITUTE OF TECHNOLOGY. Downloaded on December 29,2020 at 05:52:14 UTC from IEEE Xplore. Restrictions apply.
 2017 International Conference on Intelligent Computing,Instrumentation and Control Technologies (ICICICT)

coefficients. In order to obtain five sub bands, this method is should be such that it maximizes the interclass separation. It is
repeated. The different mother wavelets available are: “haar”, a supervised learning model where predefined class values are
“biorthogonal”, “daubechies”, “symlets” and many more. The used for training. The EEG data can also be converted to a
choice of wavelets must be such that they have a near optimal higher dimensional mapping. The mapping function can be
time-frequency localization property [12]. DWT decomposes found using Kernel functions. The most popular kernel
one dimensional time signal ‫ݔ‬ሺ‫ݐ‬ሻ as follows: function is RBF kernel. Solving the constrained optimization
’ ଵ ௧ିଶೌ ௡ problem as in (9) produces an optimal hyperplane.
‫ܹܶܦ‬ሺ‫ݔ‬ሺ‫ݐ‬ሻǢ ܽǡ ݊ሻ ൌ ‫ݔ ’ି׬‬ሺ‫ݐ‬ሻ ೌ ߖ ቀ ೌ ቁ ݀‫ݐ‬ (3)
ξଶ ଶ
where ʹ௔ ݊ and ʹ௔ are the time localization and scale ȁȁ‫ݓ‬ȁȁଶ ൅ ‫ ܥ‬σே
‹ ௜ ߦ௜ (9)
respectively, while ߖሺ‫ݐ‬ሻ denotes the mother wavelet function. ೏
௪‫א‬Թ ǡక೔ ‫א‬Թ శ

Following this the sub-band energy, relative energy and

entropy features can be obtained using (4), (5) and (6): subject to ‫ݕ‬௜ ሺ‫ݔ ் ݓ‬௜ ൅ ܾሻ ൒ ͳ െ ߦ௜ for ݅ ൌ ͳ ǥ ܰ
where C is the regularization constant ߦ௜ is the slack variable.
‫ܧ‬ሺܽሻ ൌ σ௡ ‫ܥ‬௔ଶ ሺ݊ሻ (4) This results in the dual problems
ଵ
ƒš σ௡௜ୀଵ ߙ௜ െ σ௡௜ǡ௝ୀଵ ߙ௜ ߙ௝ ‫ݕ‬௜ ‫ݕ‬௝ ‫ܭ‬ሺ‫ݔ‬௜ ǡ ‫ݔ‬௝ ሻ (10)
ாሺ௔ሻ ఈ ଶ
‫݌‬ሺܽሻ ൌ σ಼ (5) subject to Ͳ ൑ ߙ௜ ൑ ‫ܥ‬ǡ ݅ ൌ ͳǡ ǥ Ǥ ǡ ݊ǡ
ೖసభ ாሺ௞ሻ
σ௡௜ୀଵ ߙ௜ ‫ݕ‬௜ ൌ Ͳ
‫ܪ‬ሺܽሻ ൌ െ‫݌‬ሺܽሻŽ‘‰‫݌‬ሺܽሻ (6) where ߙ௜ is the Lagrange multiplier. The training samples for
 which Lagrangian multiplier is not zero are called support
™Š‡”‡ ‫ܥ‬௔  †‡‘–‡• –Š‡ ™ƒ˜‡Ž‡– ‘‡ˆˆ‹ ‹‡–• ƒ– ܽ ௧௛  vectors.
†‡ ‘’‘•‹–‹‘Ž‡˜‡Žƒ††‡‘–‡•–Š‡—„‡”‘ˆ†‹• ”‡–‡
VI. METHODOLOGY
™ƒ˜‡Ž‡–†‡ ‘’‘•‹–‹‘•Ǥ
The preprocessed signals from the DEAP dataset are used
D. Particle Swarm Optimization(PSO) for classifying human emotions in this paper. MATLAB
PSO is a population based self-adaptive search (Matrix Laboratory) is the software that we have used to
optimization technique introduced by Kennedy and Eberhart perform feature extraction and classification methods.
in 1995. It simulates the flocking of birds or schooling of fish. MATLAB is a proprietary programming language developed
The algorithm begins by randomly initializing a swarm of by MathWorks. As the signals are already down sampled and
particles in the problem domain with a certain velocity. Each filtered we perform feature extraction. The one sided power
particle's position is expressed a candidate of optimal solution spectral density is computed for all the 32 channels using N
in optimization problem. The particles search for a global point FFT with a sampling frequency of 128 Hz. MSCE is
optimum solution in the multidimensional problem space. The extracted using the Welch's averaged periodogram method
fitness value associated with every particle is optimized using with a hamming window using 128 Hz sampling frequency. A
the fitness function. In every iteration two positions are five level DWT is performed to obtain the wavelet coefficients
obtained, one being the best position of the individual particles using “db8” as the mother wavelet. Wavelet energy and
and is called the particle best (pbest), the other is the best entropy are extracted from these five subbands. Thus our
position of the entire swarm and is called the global best features consist of PSD, MSCE, Energy and Entropy. Owing
position (gbest). The velocity and the position of the particles to high dimensionality of this feature it is important to find the
are iteratively updated using (7) and (8): features which are of relevance to the classifier. Thus we
perform feature selection. PSO algorithm is chosen for
‫ݒ‬௜ ሺ‫ ݐ‬൅ ͳሻ ൌ ‫ݒ‬௜ ሺ‫ ݐ‬൅ ͳሻ‫ ݓ‬൅ ܿଵ ‫ݎ‬ଵ ሺ‫ݔ‬௣௕௘௦௧೔ ሺ‫ݐ‬ሻ െ ‫ݔ‬௜ ሺ‫ݐ‬ሻሻ ൅ optimization [13]. The classifier is trained based on these
ܿଶ ‫ݎ‬ଶ ሺ‫ݔ‬௚௕௘௦௧ ሺ‫ݐ‬ሻ െ ‫ݔ‬௜ ሺ‫ݐ‬ሻሻ (7) crucial features. SVM classifier is used for classification. RBF
kernel is selected because of its high learning capacity. The
‫ݔ‬௜ ሺ‫ ݐ‬൅ ͳሻ ൌ ‫ݔ‬௜ ሺ‫ݐ‬ሻ ൅ ‫ݒ‬௜ ሺ‫ ݐ‬൅ ͳሻ (8) feature extraction, selection and classification procedure was
further repeated for different channel combinations. The
where x is the position vector, v is the velocity vector, ‫ݐݏܾ݁݌‬ results of the classifier with and without optimization are
is the local best position of the particle, ܾ݃݁‫ݐݏ‬is the global compared.
best position of the swarm, ܿଵ and ܿଶ are acceleration
constants and ‫ݎ‬ଵ and ‫ݎ‬ଶ are two random values uniformly VII. RESULTS AND DISCUSSION
distributed between [0,1]. The stopping condition is when a
maximum velocity is reached or if a maximum number of Table I summarizes the classification accuracy using an
iterations are completed. SVM classifier with RBF kernel and a 10 fold cross validation
for different electrode combinations. It is found that the
V. SVM CLASSIFIER classification accuracies vary upon choosing different
SVM is a discriminative classifier formally defined by a combinations of electrodes. A maximum classification
separating hyperplane which can be effectively used to accuracy of 74.062 % was obtained using the 16 electrode
classify a high dimensional feature set. The hyper plane combination. It is also observed that the highest valence and

978-1-5090-6106-8/17/$31.00 ©2017 IEEE 1599

Authorized licensed use limited to: M S RAMAIAH INSTITUTE OF TECHNOLOGY. Downloaded on December 29,2020 at 05:52:14 UTC from IEEE Xplore. Restrictions apply.
 2017 International Conference on Intelligent Computing,Instrumentation and Control Technologies (I

arousal accuracies of 80% and 85% respectively are obtained VIII. CONCLUSION
using the five electrode combination (P7, P3, PZ, T7, T8). The preprocessed EEG signals from the DEAP dataset are
Table II summarizes the classification accuracies for the four successfully classifies into four discrete emotional states based
discrete classes namely happy, angry, sad and relaxed. on the Russell’s circumplex model. The core element of this
The highest accuracy is obtained using the sixteen electrode paper is PSO which optimizes the classifier thus providing
combination but sufficiently high classification accuracy is sufficiently higher classification accuracy. Also adding MSCE
obtained using the five electrode combination (P7, P3, PZ, T7, led to a better feature set as it estimates the coherence between
T8) thereby reducing the number of electrodes required. various frequency domain signals. Moreover after
experimenting with different electrode combinations we
TABLE I. ACCURACY USING DIFFERENT ELECTRODE COMBINATIONS
observed that a good classification accuracy of 70.625% was
obtained using a reduced set of 5 electrodes P7, P3, PZ, T7
and T8.
Accuracy %
Electrode
Combination Valence Arousal Overall REFERENCES

FP1, FP2 46.153 65.625 46.875 [1] R. Plutchik, “The Nature of Emotions Human emotions have deep
F3, F4, FP1, evolutionary roots, a fact that may explain their complexity and provide
75.937 72.187 57.812 tools for clinical practice”, American scientist, 89(4), pp.344-350, 2001.
FP2, FPZ
P7, P3, PZ 71.25 75.937 55.312 [2] D. Garrett, D.A. Peterson, C. W. Anderson and M. H Thaut,
“Comparison of linear, nonlinear, and feature selection methods for EEG
P7, P3, PZ, T7, signal classification”, IEEE Transactions on Neural Systems and
80 85 70.625
T8 Rehabilitation Engineering, 11 (2), pp.141-144, 2003.
P7, P3, PZ, [3] M. Soleymani, P. Maja and Thierry P. "Multimodal emotion recognition
PO3,O1,CP2, 76.875 70.312 57.187 in response to videos." IEEE transactions on affective computing 3.2,
C4 pp. 211-223, 2012.
F3, F4, FP1,
FP2, F7, F8, [4] M. Murugappan , “Human Emotion Classification using Wavelet
Transform and KNN”, International Conference on Pattern Analysis and
FTC1, FTC2,
84.375 83.437 74.062 Intelligent Robotics 28-29, June 2011.
C3, C4, O1,
O2, T7, T8, [5] C.J. Tu, L.Y. Chuang, J.Y. Chang, and C.H. Yang, “Feature Selection
P3, P4 using PSO-SVM,” International Journal of Computer Science (IAENG),
vol.8, no. 3, 2007
TABLE II. CLASSIFICATION ACCURACY [6] M Sreeshakthy and J Preethi,"Classification of Human Emotion from
Deap EEG Signal Using Hybrid Improved Neural Networks with
Cuckoo Search." BRAIN. Broad Research in Artificial Intelligence and
Accuracy % Neuroscience 6.3-4 , pp. 60-73,2016.
Electrode
Combination Class 1 Class 2 Class 3 Class 4 [7] S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi,
Happy Sad Angry Relaxed T. Pun, A. Nijholt, and I. Patras, “DEAP: A database for emotion
analysis using physiological signals,”IEEE Transactions on Affective
FP1, FP2 46.153 37.878 45.833 58.730 Computing, vol. 3, no. 1, pp. 18–31, 2012.
[8] J. A. Russell, “A circumplex model of affect”, Journal Of Personality
F3, F4, FP1, and Social Psychology, 39, pp. 1161-1178, 1980.
57.812 56.944 54.717 61.971
FP2, FPZ [9] X. W. Wang, N. Dan and L. Bao-Liang, "EEG based emotion
P7, P3, PZ 60.483 50 47.169 57.746 recognition using frequency domain features and support vector
machines", International Conference on Neural Information Processing,
P7, P3, PZ, T7,
73.387 66.67 64.15 74.647 pp. 734-743, Springer Berlin Heidelberg, 2011.
T8
P7, P3, PZ, [10] P. D. Welch, “The Use of Fast Fourier Transform for the Estimation of
PO3,O1,CP2, 60.483 59.722 49.056 54.929 Power Spectra: A Method Based on Time Averaging Over Short,
C4 Modified Periodograms,” IEEE Transactions on Audio and
F3, F4, FP1, Electroacoustics, vol. 15, pp. 70–73, 1967.
FP2, F7, F8, [11] R. Khosrowabadi, H. C. Quek, A. Wahab, and K. K. Ang. "EEG-based
FTC1, FTC2, emotion recognition using self-organizing map for boundary detection."
74.193 69.444 77.358 76.056 In Pattern Recognition (ICPR), 20th International Conference on, pp.
C3, C4, O1,
O2, T7, T8, 4242-4245. IEEE, 2010.
P3, P4 [12] M.Murugappan, R.Nagarajan, and S.Yaacob, "Comparison of different
wavelet features from EEG signals for classifying human emotions." In
Industrial Electronics & Applications,ISIEA 2009. IEEE Symposium on,
vol. 2, pp. 836-841. IEEE, 2009.
[13] Subasi, Abdulhamit. "Classification of EMG signals using PSO
optimized SVM for diagnosis of neuromuscular disorders." Computers
in biology and medicine 43, no. 5, pp. 576-586, 2013.

978-1-5090-6106-8/17/$31.00 ©2017 IEE 1600

Authorized licensed use limited to: M S RAMAIAH INSTITUTE OF TECHNOLOGY. Downloaded on December 29,2020 at 05:52:14 UTC from IEEE Xplore. Restrictions apply.