Consumer Grade Brain Sensing For Emotion Recognition
Consumer Grade Brain Sensing For Emotion Recognition
Abstract—For several decades, electroencephalography (EEG) Over the years, emotion studies have broadened from psycho-
has featured as one of the most commonly used tools in emotional physiological focuses to engineering applications [2], and
state recognition via monitoring of distinctive brain activities. An
arXiv:1810.04582v4 [eess.SP] 9 Aug 2019
number of which are freely available to the public. One of the usability of an open-source consumer grade EEG amplifier,
first publicly accessible datasets, MAHNOB-HCI, was created OpenBCI, in emotion recognition application. In particular,
to meet the growing demand from the scientific community for by adopting the same paradigm and classification algorithm
multifaceted information. The dataset contains EEG signals, as used by high-end EEG works, we were able to appraise the
physiological responses, facial expressions, and gaze data prediction accuracies of OpenBCI-derived data in a systematic
collected during participants’ video viewing [8]. The affective manner against previous studies and found them to be more or
clips were short section cuts from a collection of commercial less comparable [8], [9], [12]. We developed a classification
films. The experimental design was built upon a preliminary model to predict whether the recorded EEG data had a high
study whereby online volunteers reported the emotions expe- or low level of valence and arousal. The algorithm essentially
rienced during their watching of the clips. In the main study, imparts simple feature extraction (e.g., power spectral density
the participants were instructed to submit a subjective rating (PSD)) and the classifier was constructed with a support
on arousal, valence, dominance, and content predictability. vector machine (SVM) architecture. In sum, our classification
Another dataset called Database for Emotion Analysis using results appeared on par if not better when compared to the
Physiological Signals or DEAP was similarly motivated; it results from benchmark datasets that were generated from
contains brainwave recordings from EEG in combination with high-end EEG devices. Our two principal contributions can
peripheral physiological signals [9]. The stimulus of choice be summarized as follows:
was an assortment of music videos that were pre-selected • We propose a robust stimulus selection method and
based on the demographic background of the participants, validate a clustering approach for emotion labeling.
most of whom were European. These music videos were • We provide concrete evidence for the capability of con-
retrieved from a website that collects songs and music videos sumer grade OpenBCI in emotion recognition study by
tagged with different emotion descriptors by users. Each of quality assessing our performance accuracies against re-
the music videos received a score from the participants on sults from public repositories.
arousal, valence, dominance, liking, and familiarity. The remainder of this paper provides an overview of the rising
As opposed to the previously mentioned datasets, a presence of consumer grade EEG devices in emotion studies
multimodal dataset built for the decoding of user physiological (Section II), methodology in Section III, results in Section IV,
responses to affective multimedia content or DECAF discussions in Section V, and conclusion in Section VI.
compared the signals obtained from EEG to the recordings
from ELEKTA Neuromag, a Magnetoencephalogram (MEG)
sensor [10]. Video selection was reliant on the affective II. C ONSUMER G RADE EEG FOR E MOTION R ECOGNITION
notations reported by volunteers prior to the study. In the According to our literature survey on recently published ar-
data collection phase, participants were told to rate the music ticles using “Low-Cost EEG Emotion” as a search phrase, the
videos and movie clips in terms of arousal, valence, and majority of studies used EEG headsets from EMOTIV EPOC+
dominance. Other types of measurable signals that correlated [18], while others used EMOTIV INSIGHT, MYNDPLAY
with implicit affective states were simultaneously monitored [19], NeuroSky [20], and MUSE [21]. Only a few studies
and collected with horizontal Electrooculogram (hEOG), on emotion recognition used OpenBCI [22]. Table I presents
Electrocardiogram (ECG), Near-Infra-Red (NIR) facial a comparison between a brand-new consumer grade EEG by
videos, and trapezius-Electromyogram (tEMG). Constructed the name of OpenBCI and other consumer grade EEGs on the
in 2015, SJTU Emotion EEG Dataset or SEED comprises market. Each product has been scientifically validated against
EEG, ECG, EMG, EOC, and skin resistance (SC) data high-end EEGs by measuring the established brain responses.
acquired from 15 Chinese students [11]. Fifteen emotion- Three studies validated EPOC+ against EEG from ANT Neuro
charged clips were selected from a pool of popular locally [23], Neuroscan [24], and gtec [25]. The studies included
produced Chinese films. The participants were directed to the measurement of P300, Event-Related Potential (ERP), and
categorize the clips as either positive, neutral, or negative. emotion, respectively. Frontal EEGs recorded from NeuroSky
However, the classification of emotions in this dataset did and MUSE were validated with two baseline EEGs named B-
not include valence and arousal as labels. This study was an Alert [26] and Enobio [27]. MUSE was also validated with
extension from the earlier study by the same research group the EEG from Brain Products GmbH [28] for ERP research.
where movie clips were being scored on three attributes i.e., Recently, one research group reported the performance of
valence, arousal, and dominance [6]. The most recent dataset OpenBCI based on Texas Instrument ADS1299 (biopotential
is possibly DREAMER, created using consumer grade EEG amplifier) using movement-related cortical potential measure-
and ECG. The EEG system used was an Emotiv EPOC ment. In comparison to the product from Neuroscan [24], there
which is famous for its easy-to-wear design and wireless were no statistically significant differences between the two
headset [12]. To gather the relevant data, the team instructed makes.
their recruits to fill in the SAM form to rate their levels of Research on emotion-related topics that adopt consumer
arousal, valence, and dominance. Short videos clips were grade EEG can be separated into two domains depending
chosen as the means of emotion elicitation and the emotions on the types of emotion-eliciting stimuli. The first domain
most likely to be invoked from these clips were predetermined. consists of audio-visual stimulation (video clips). Two studies
[12], [29] set out to construct emotion datasets for public
The key objective of our investigation was to evaluate the release and develop recognition algorithms for prospective
3
Product Name Price [USD] Sampling Rate [Hz] No. of Channels Open Source Raw Data Scientific Validation
OpenBCI 750/1800 250 8/16 Yes Yes MRCPs [13]
EMOTIV EPOC+ 799 256 14 SDK Yes* P300 [14], ERP [15], emotion [12]
EMOTIV INSIGHT 299 128 5 SDK Yes* -
Myndplay Myndband 200 512 3 SDK* Yes -
NeuroSky MindWave Mobile 99 512 1 Yes Yes frontal EEG [16]
InteraXon MUSE 199 220 4 SDK Yes frontal EEG [16], ERP [17]
applications. One research team integrated EMOTIV EPOC+ such as an investigation involving 103-day single channel EEG
with a bracelet sensor [30] and eye tracker [31] to produce data on emotion recognition in daily life [49]. Nowadays,
a consumer grade emotion recognition system [32]. An evo- fundamental knowledge on emotion recognition or affective
lutionary computation was then proposed as a competitive computing using brain wave activity has been applied in broad
algorithm in emotion recognition and EMOTIV INSIGHT was areas such as studies of people with depression [50], stress in
used for performance evaluation [33]. Furthermore, MUSE and construction workers [51], and interactions between ambient
certain other physiological datasets were used in another work temperature in building and the occupant [52].
involving boredom detection during video clip viewing [34]. As previously mentioned, this paper describes a feasibility
Data collected from the MUSE headband were also shown study of emotion recognition by OpenBCI. In recent years,
to be able to predict the affective states and their magnitudes there have been a few related works, one of which demon-
[35]. As an example of its practical application, a real-time strated the usefulness of OpenBCI as a biofeedback instrument
emotion recognition framework was recently proposed, using (EEG, EMG) for an application titled IBPoet [53]. Three
EMOTIV EPOC+ to record participants’ brainwaves as they parties participated in the IBPoet demonstration: the reader,
were being shown Chinese film excerpts [36]. The second the readee, and the audience. Once the reader had relayed
domain involves auditory stimulation, most often music, as the selected poem, the readee sent biofeedback (emotional
the means to invoke desired emotions. The use of EMOTIV responses) to the reader via a vibration band and heat glove.
EPOC+ was demonstrated for the automatic detection of The reader and the system adapted according to the emotional
brain responses from three types of music (neutral, joyful, responses to give the audience a greater feeling of pleasure.
and melancholic). EEG connectivity features were reported to Similarly, another research group used the same device to
play an essential role in the recognition tasks [37]. Another evaluate and practice oral presentation [54]. The other findings
study, using the same device, observed a correlation of EEG are related to typical research on emotion recognition, but
with classical music rhythm [38], [39]. Music therapy is one the numbers of experimental subjects were very small (some
of the applications emanating from music-elicited emotion have several subjects) [55]–[58]. Moreover, EEG software for
[40]. Moreover, in the study of auditory-related emotion, one automated emotion recognition to support consumer grade
research group reported the development of envisioned speech devices, including OpenBCI, was proposed in late 2018 [59].
recognition using EEG [41]. Thus, it could be inferred that the demand for OpenBCI is
Motivated by the findings of the aforementioned research increasing. Therefore, this study focuses on the feasibility of
domains on emotion-related works, researchers from other using OpenBCI in emotion recognition research.
fields have also used consumer grade EEG devices. Examples
of such studies are provided in this paragraph to enable the III. M ETHODOLOGY
reader to get a sense of the impact of consumer grade devices. This section begins with an experiment to select effective
Two papers have reviewed past literature on the study of film trailers for emotion elicitation (Experiment I). Consumer
human behavior using brain signals for consumer neuroscience grade EEG and peripheral physiological sensors are introduced
and neuromarketing applications [42], [43]. In addition, a in Experiment II. The selected trailers from Experiment I
consumer-related research study showcased the effect of color are used to invoke emotions, and corresponding EEG and
priming on dress shopping, measured by EEG [44]. Another peripheral physiological signals are recorded and stored for
group reported the feasibility of using asymmetry in frontal subsequent analysis. All experiments fully abide by the 1975
EEG band power (especially the Alpha band) as a metric Declaration of Helsinki (revised in 2000), and have been
for user engagement while reading the news [45]. Band granted ethical approval by the Internal Review Board of
power asymmetry was also introduced as a study feature Chiang Mai University, Thailand.
on the effects of meditation on emotional responses [46].
Some groups explored simultaneous EEG recordings from
multiple subjects such as the research on synchronized-EEGs A. Experiment I: Affective Video Selection
while students were studying in the same classroom [47] Internet Movie Database (IMDb) provides categorical labels
and similar research on the emotional engagement between a for film trailers based on the main genres including drama,
performer and multiple audiences [48]. Other research groups comedy, romance, action, sci-fi, horror, and mystery. We
have studied longitudinal EEG data from individual subjects simplified these categories into three umbrellas: Happiness,
4
1 2 3 4 5
Select film trailers Five participants Randomly split into four groups 200 participants view trailers Each participant assesses his/her emotions
categorized by IMDB select 120 famous film trailers (30 trailers per group) from one out of the four groups for each trailer (H, F, E — 1-9)
clip 01 / 30
A B H
F
E
NEXT
C D
1 2 3 4
Gather all H, F, E scores Perform k-mean clustering (k=3) Remove data points which are not in Select the first 20 closest points
from all trailers and participants among those points their majority cluster for every trailer To the centroids of each cluster output
Happiness (H)
Happiness (H)
Happiness (H)
7 7 7 7
6 6 6 6
5 5 5 5 20 trailers from green
4 4 4 4
3 3 3 3
2
1
2
1
2
1
2
1 20 trailers from red
9 9 9 9
8 8 8 8
7 6 9 7 6 9 7 6 9 7 6 9
5 7 8 5 7 8 5 7 8 5 7 8
4 3 5 6 4 3 5 6 4 3 5 6 4 3 5 6
Fea 2 3 4 Fea 2 3 4 Fea 2 3 4 Fea 2 3 4
r (F) 1 1 2
(E) r (F) 1 1 2
(E) r (F) 1 1 2
(E) r (F) 1 1 2
(E)
Excitement Excitement Excitement Excitement
Fig. 1: Experiment I starts with (a) random selection of 120 IMDb movie trailers by five individuals. Two-hundred participants
are then presented 30 trailers each and afterward asked to assess how the trailer makes them feel. (b) K-means clustering is
performed to identify trailers that are effective at inducing emotions in viewers.
input personal information self emotional assessment self emotional assessment self emotional assessment Rate liking scores for each clip
Fig. 3: Each experimental run begins with the acquisition of demographic data including participants personal information,
hours of sleep, level of tiredness, and favorite movie genre. Video clips are played in full-screen mode and a display of emotion
self-assessment form is launched at the end of each clip. Screenshots shown here are of six movies by the titles in [60]–[65]
of the international standard 10–20 system for EEG electrode Arousal, Valence, Happiness, Fear, and Excitement, on a
placement. In addition to EEG, we also recorded peripheral scale of 1 to 9. Afterward, we checked their answers to
physiological signals from a wearable Empatica4 or E4 device ensure that they understood correctly. Subsequently the actual
[66], which is equipped with multiple sensors, including experiment was launched, as shown in Figure 3. Firstly, we
electrodermal activity (EDA), skin temperature (Temp), and collected personal information (age, gender, hours of sleep,
blood volume pulse (BVP) with their derivatives (heart rate, level of tiredness, and favorite movie genre). Secondly, a
inter-beat interval, etc.). Interestingly, E4 was recently utilized film trailer was played for 1 minute. Then, we presented a
by a research team to record electrodermal activity (EDA) in screen displaying the self-assessment questionnaire similar to
25 healthy participants in the study of arousal and valence one featured in the trial run. Finally, another film trailer was
recognition [67]. played, followed by the questionnaire and the process repeated
2) Participants: Forty-three healthy individuals aged be- for all 15 clips. More specifically, a fixed set of 9 (out of 15)
tween 16 and 34 (21 male and 22 female) were recruited as clips were played to all 43 participants; these included the top
volunteers for the study. They were fitted with two monitoring three clips from each cluster from Experiment I. The other six
devices for simultaneous recording: OpenBCI (EEG) and video clips were selected randomly: two per cluster.
Empatica4 (E4). E4 was strapped on the wrist with extension 4) Feature Extraction: In order to obtain stable and perti-
cables and gel electrodes attached (Kendall Foam Electrodes) nent emotional responses from the EEG and peripheral physi-
to measure EDA on the middle knuckle of the index and ological signals, we started the recording after each movie clip
middle finger. During the main experiment, the participants had been played for two seconds and stopped two seconds be-
were instructed to place their chin on the designated chin fore the clip ended. This means that on the whole, 56 seconds
rest and stay as still as possible in order to minimize artifact of signals were collected for a single video clip. Typical EEG
interference. We also strove to maintain the impedance below pre-processing steps were applied including notch filtering
5 kΩ for EEG. The experimental setup is depicted in Figure 2. using iirnotch, common average reference (CAR) to find the
3) Experimental Protocol: We developed a software to reference signal for all electrode channels, and independent
present the emotion-inducing video clips sequentially, and component analysis (ICA) to remove artifact components.
recorded synchronous EEG and peripheral physiological sig- Even if there was no EOG present, ICA components were
nals with two wearable devices (see Figure 2). Although we subject to inspection. On average, zero to one component with
intended to elicit a specific emotion from the clip as defined by characteristics most similar to those of EOG was removed in
IMDb tags, each participant might have experienced emotions keeping with the manual provided by MNE [68]. Both CAR
that were incongruent with the expectation. Owing to this, and ICA were implemented using MNE-python package [69].
when labelling the data, each participant was instructed to Conventional feature extraction, as shown in Table II, was
rate his/her own emotional reaction to the clip while being computed from pre-processed EEG, EDA, BVP, and Temp.
blinded to its IMDb tags. To avoid potential misunderstanding, These features were based on previous works [8], [9], [12]
before commencing the main experiment, we methodically and included as the baseline for comparison to the results
described the procedures and the meaning of every emotion of this study. To extract features within a specified range of
score. Additionally, an example clip, which was not used in the EEG frequencies, we applied lfilter (from SciPy package) for
actual experiment, was shown during a mock trial to test the bandpass filtering [70].
participants’ understanding. We then presented a questionnaire 5) Emotion Recognition: For this part, the investigation
for them to rate their experienced emotions pertaining to was divided into three subtasks, as detailed in the bullet
6
TABLE II: Description of features that are extracted from EEG • Low-High: Valence/Arousal with Clustering
and Empatica signals. We conducted GMM and K-means clustering using V
and A scores from the participants in order to select the
Signal Extracted Features
most suitable clustering method. DB-index was used as
EEG(32) θ (3–7 [Hz]), α (8–13 [Hz]), β (14–29 [Hz]) and γ a metric and it turned out that K-means clustering with 4
(30–47 [Hz]) power spectral density for each channel
clusters was the best one. According to four combinations
EDA(21) average skin resistance, average of derivative, aver- of V and A models in a conventional emotion-related
age of derivative for negative values only, proportion study (low (L) or high (H) V and A scores), we labeled
of negative samples in the derivative vs all samples,
number of local minima, average rising time, 14 samples in the four groups stratified by K-means with
spectral power in the 0–2.4 [Hz] bands, zero crossing LVLA, LVHA, HVLA, and LVHA. The input features
rate of skin conductance slow response 0–0.2 [Hz], were identical to the previous subsection.
zero crossing rate of skin conductance very slow
response 0–0.08 [Hz] • EEG Electrode Channels and Frequency Bands
In this subtask, the electrode setups were examined for
BVP(13) average and standard deviation of HR, HRV, and in- prospective design of a user-friendly EEG device capable
ter beat intervals, energy ratio between the frequency
bands 0.04–0.15 [Hz] and 0.15–0.5 [Hz], spectral of emotion recognition. Here, 3 or 4 out of 8 electrode
power in the bands 0.1–0.2 [Hz], 0.2–0.3 [Hz], 0.3– channels were strategically chosen to explore options
0.4 [Hz], low frequency 0.01–0.08 [Hz], medium for hardware design. The process began by studying the
frequency 0.08–0.15 [Hz] and high frequency 0.15–
0.5 [Hz] components of HRV power spectrum sagittal line of the head (Fz , Cz , Pz Oz ), and another
eight sets of channels were then created (F P1 , F P2 ,
Temp(4) average, average of its derivative, spectral power in Fz ), (F P1 , F P2 , Cz ), (F P1 , F P2 , Cz ), (F P1 , F P2 , Pz ),
the bands 0–0.1 [Hz] and 0.1–0.2 [Hz]
(F P1 , F P2 , Oz ), (T3 , T4 , Fz ), (T3 , T4 , Cz ), (T3 , T4 , Pz )
and (T3 , T4 , Oz ). These were taken from a combination
of either the temporal or frontal lobe and one channel
points below. All analyses were based on binary classification from the sagittal line. In addition, we sought to pinpoint
(low-high valence and arousal). The first analysis relied on a the important frequency bands within the EEG signals,
straightforward threshold setting for labeling of two separable provided the four fundamental bands: θ (3–7 [Hz]), α (8–
classes, while the second and third relied on the use of K- 13 [Hz]), β (14–29 [Hz]), and γ (30–47 [Hz]). For binary
means clustering technique. One sample point correlated to a classification (low-high V and A), we performed labeling
recording from one participant in response to the short video in a similar manner to Low-High: Valence/Arousal with
clip; there were 645 (43 participants × 15 clips) sample points Clustering and compared the results among the sets of
overall. channels and frequency bands.
• Low-High: Valence/Arousal with Threshold
We labeled each sample on the basis of associated V and We implemented leave-one-clip-out cross-validation using
A scores and manually set the threshold at 4.5 (midpoint Support Vector Machine (SVM), as illustrated in Figure 4.
between 1 to 9) for binary classification. In essence, a Since a fixed set of 9 video clips were commonly seen by
sample with V or A score lower than 4.5 was labeled the participants (of all 15 videos watched by each), nine-fold
as low V or A, and vice versa. Two binary classification cross-validation was conducted. In each fold, one clip was set
tasks were then carried out, high vs low valence and high aside as a test sample and the rest were used as a training set.
vs low arousal. Table II displays the set of input features We normalized all features by using MinMaxScaler to scale
for the model, the total being 70 features. them into a common range [71]. In the training session, the
7
TABLE IV: Comprehensive list of 60 affective video titles featured in Experiment II to allow monitoring of emotionally
relevant EEG and peripheral physiological signals. Means and standard deviations of quantified attributes (Valence, Arousal,
Happiness, Fear, Excitement) are listed in adjacent columns. ’Samp’ (short for sample) denotes the number of participants
having seen the particular clip and submitted subjective ratings.
ID Movie Title Affective Tags (IMDb) Valence Arousal Happiness Fear Excitement Samp
1 The Help Drama 4.28 ± 2.08 3.34 ± 1.96 4.41 ± 2.12 1.26 ± 0.37 2.56 ± 1.66 82
2 Star Wars: The Last Jedi Action, Adventure, Fantasy, Sci-Fi 4.98 ± 2.15 4.36 ± 2.08 4.06 ± 2.16 1.79 ± 1.00 5.09 ± 2.12 81
3 Suicide Squad Action, Adventure, Fantasy, Sci-Fi 4.73 ± 2.02 4.12 ±1.90 4.36 ± 2.18 1.73 ± 1.41 4.37 ± 2.14 81
4 Pacific Rim Action, Adventure, Sci-Fi 4.10 ± 2.03 3.81 ± 2.16 3.93 ± 2.04 1.31 ± 0.52 4.25 ± 2.12 44
5 War for the Planet of the Apes Action, Adventure, Drama, Sci-Fi, Thriller 4.73 ± 2.06 4.04 ± 2.20 2.57 ± 1.90 2.67 ± 1.76 4.83 ± 2.08 41
6 God Help the Girl Drama, Musical, Romance 4.38 ± 2.45 3.09 ± 2.02 5.00 ± 2.45 1.27 ± 0.47 3.01 ± 1.85 43
7 Rogue One: A Star Wars Story Action, Adventure, Sci-Fi 5.09 ± 1.78 4.65 ± 2.15 3.34 ± 2.17 2.04 ± 1.23 5.28 ± 2.27 42
8 Blade Runner 2049 Drama, Mystery, Sci-Fi 4.44 ± 2.47 4.34 ± 2.38 3.15 ± 2.02 2.39 ± 1.73 4.81 ± 2.29 44
9 Hope Springs Comedy, Drama, Romance 4.78 ± 2.46 3.56 ± 2.12 5.09 ± 2.34 1.19 ± 0.17 2.84 ± 1.89 45
10 Ghost in the Shell Action, Drama, Sci-Fi 4.77 ± 2.10 4.28 ± 2.27 3.41 ± 2.13 2.07 ± 1.57 5.01 ± 2.27 47
11 Point Break Action, Crime, Sport 4.65 ± 2.33 4.49 ± 2.40 3.31 ± 2.33 2.08 ± 1.41 5.12 ± 2.56 40
12 The Hunger Games Adventure, Sci-Fi, Thriller 5.42 ± 2.26 4.76 ± 2.18 3.66 ± 2.09 2.06 ± 1.34 5.38 ± 2.05 42
13 Crazy, Stupid, Love. Comedy, Drama, Romance 4.82 ± 2.54 4.01 ± 2.52 5.12 ± 2.64 1.21 ± 0.25 3.06 ± 2.35 43
14 Arrival Drama, Mystery, Sci-F 4.84 ± 2.22 4.82 ± 2.28 3.28 ± 2.03 2.64 ± 1.90 5.66 ± 2.40 44
15 Mr. Hurt Comedy, Romance 4.50 ± 2.21 3.59 ± 2.20 4.66 ± 2.19 1.23 ± 0.21 2.41 ± 1.43 42
16 American Assassin Action, Thriller 4.19 ± 2.18 4.80 ± 2.33 2.90 ± 1.88 2.48 ± 1.92 5.03 ± 2.36 43
17 G.I. Joe: Retaliation Action, Adventure, Sci-Fi 4.69 ± 2.39 4.11 ± 2.31 3.46 ± 2.09 1.55 ± 0.93 5.02 ± 2.54 48
18 Beginners Comedy, Drama, Romance 4.42 ± 2.45 3.08 ± 2.18 4.97 ± 2.20 1.23 ± 0.27 2.42 ± 1.48 44
19 Open Grave Horror, Mystery, Thriller 3.70 ± 1.90 4.70 ± 2.30 1.90 ± 1.14 4.03 ± 2.40 4.55 ± 2.46 43
20 Flipped Comedy, Drama, Romance 5.05 ± 2.69 3.75 ± 2.40 5.43 ± 2.48 1.16 ± 0.08 2.88 ± 2.12 44
21 The Choice Drama, Romance 4.58 ± 2.11 4.12 ± 1.98 4.63 ± 2.16 1.43 ± 0.68 3.24. ± 1.71 82
22 Danny Collins Biography, Comedy, Drama 4.54 ± 2.20 3.55 ± 2.07 5.01 ± 2.09 1.17 ± 0.09 2.76 ± 1.68 82
23 The Big Sick Comedy, Drama, Romance 4.44 ± 2.10 3.21 ± 1.80 4.56 ± 2.15 1.28 ± 0.35 2.93 ± 1.90 86
24 Monsters University Animation, Adventure, Comedy 5.55 ± 2.07 4.01 ± 2.12 6.15 ± 2.12 1.24 ± 0.23 4.44 ± 2.14 47
25 Kung Fu Panda 3 Animation, Action, Adventure 6.11 ± 2.17 4.46 ± 2.31 6.44 ± 2.37 1.25 ± 0.24 4.14 ± 2.11 47
26 Baby Driver Action, Crime, Drama 5.29 ± 2.10 4.65 ± 2.27 5.19 ± 2.32 1.53 ± 0.88 5.30 ± 2.28 44
27 The Good Dinosaur Animation, Adventure, Comedy 6.43 ± 2.15 4.63 ± 2.33 6.47 ± 2.18 1.19 ± 0.41 4.22 ± 2.07 43
28 About Time Comedy, Drama, Fantasy 5.25 ± 2.60 4.29 ± 2.68 5.97 ± 2.32 1.22 ± 0.34 4.25 ± 2.24 43
29 Ordinary World Comedy, Drama, Music 4.88 ± 1.72 3.93 ± 1.72 5.47 ± 1.68 1.22 ± 0.20 3.45 ± 1.98 49
30 Lion Biography, Drama 5.36 ± 2.30 4.62 ± 2.65 5.01 ± 2.49 1.79 ± 1.25 3.93 ± 2.35 48
31 Shrek Forever After Animation, Adventure, Comedy 5.87 ± 2.12 3.85 ± 2.37 6.29 ± 2.29 1.26 ± 0.29 4.35 ± 2.49 44
32 Chappie Action, Crime, Drama 5.47 ± 2.26 4.43 ± 2.20 4.54 ± 2.45 1.69 ± 0.73 5.31 ± 2.31 47
33 Guardians of the Galaxy Vol. 2 Action, Adventure, Sci-Fi 6.15 ± 2.40 4.61 ± 2.34 5.85 ± 2.40 1.44 ± 1.01 5.56 ± 2.50 48
34 The Intern Comedy, Drama 6.34 ± 2.02 5.06 ± 2.13 6.31 ± 1.98 1.23 ± 0.39 3.74 ± 2.34 42
35 La La Land Comedy, Drama, Music 5.44 ± 2.24 4.09 ± 2.37 5.55 ± 2.29 1.49 ± 1.28 3.15 ± 1.99 47
36 Ice Age: Collision Course Animation, Adventure, Comedy 6.38 ± 2.36 4.96 ± 2.40 6.92 ± 1.94 1.21 ± 0.26 4.87 ± 2.36 43
37 Frozen Animation, Adventure, Comedy 5.88 ± 2.40 4.17 ± 2.38 6.31 ± 2.41 1.24 ± 0.40 4.35 ± 2.36 47
38 Transformers: The Last Knight Action, Adventure, Sci-Fi 4.57 ± 2.06 4.18 ± 1.96 4.10 ± 2.33 1.91 ± 1.22 4.87 ± 1.97 42
39 Divergent Adventure, Mystery, Sci-Fi 5.87 ± 1.81 4.87 ± 2.00 4.75 ± 2.27 1.95 ± 1.34 5.84 ± 2.05 49
40 Why Him? Comedy 5.85 ± 2.24 4.60 ± 2.40 6.03 ± 2.30 1.25 ± 0.39 4.06 ± 2.44 43
41 The Boy Horror, Mystery, Thriller 3.85 ± 2.09 4.92 ± 1.97 1.78 ± 1.06 5.21 ± 2.23 5.01 ± 2.16 82
42 Jigsaw Crime, Horror, Mystery 4.04 ± 2.14 4.68 ± 2.15 2.07 ± 1.37 4.65 ± 2.18 4.91 ± 2.18 86
43 Shutter Horror, Mystery, Thriller 3.53 ± 2.20 4.71 ± 2.25 1.68 ± 0.97 5.23 ± 2.34 4.63 ± 2.29 81
44 Ladda Land Horror 4.61 ± 2.20 4.81 ± 2.23 1.95 ± 1.47 5.62 ± 2.13 4.88 ± 2.07 42
45 No One Lives Horror, Thriller 4.20 ± 2.04 4.84 ± 2.28 1.94 ± 1.35 4.97 ± 2.56 5.07 ± 2.33 47
46 Tales from the Crypt Horror 3.83 ± 2.22 4.41 ± 2.21 2.21 ± 1.88 4.67 ± 2.36 4.68 ± 2.41 44
47 Orphan Horror, Mystery, Thriller 4.07 ± 2.35 5.18 ± 2.11 1.85 ± 1.40 5.11 ± 2.15 4.68 ± 2.27 40
48 Unfriended Drama, Horror, Mystery 4.34 ± 2.57 5.40 ± 2.57 1.98 ± 1.93 5.34 ± 2.53 5.37 ± 2.55 43
49 Poltergeist Horror, Thriller 4.13 ± 2.44 5.28 ± 2.55 1.91 ± 1.70 5.85 ± 2.36 5.20 ± 2.60 43
50 Jeruzalem Horror 4.20 ± 2.12 4.82 ± 2.14 2.02 ± 1.42 5.00 ± 2.23 4.90 ± 2.27 49
51 Leatherface Crime, Horror, Thriller 3.92 ± 2.11 4.77 ± 2.39 1.89 ± 1.27 4.93 ± 2.53 4.90 ± 2.50 47
52 The Babadook Drama, Horror 3.62 ± 2.06 4.84 ± 2.19 1.67 ± 1.05 5.34 ± 2.11 4.74 ± 2.17 43
53 Oculus Horror, Mystery 4.12 ± 2.11 5.81 ± 1.90 1.76 ± 1.29 6.07 ± 1.81 5.36 ± 2.28 42
54 The Witch Horror, Mystery 3.69 ± 2.17 4.69 ± 2.44 1.89 ± 1.38 5.12 ± 2.40 4.54 ± 2.37 42
55 Trick ’r Treat Comedy, Horror, Thriller 4.50 ± 2.27 5.59 ± 2.30 1.93 ± 1.42 5.68 ± 2.26 5.55 ± 2.34 42
56 The Woman in Black Drama, Fantasy, Horror 4.23 ± 2.29 4.67 ± 2.27 1.94 ± 1.54 5.21 ± 2.33 4.79 ± 2.16 42
57 The Possession Horror, Thriller 4.83 ± 2.47 5.32 ± 2.29 1.82 ± 1.44 5.90 ± 2.42 5.66 ± 2.38 42
58 Crimson Peak Drama, Fantasy, Horror 3.93 ± 2.15 4.79 ± 2.33 2.33 ± 1.71 4.81 ± 2.43 5.10 ± 2.16 43
59 Program na winyan akat Horror, Thriller 3.95 ± 2.11 4.98 ± 2.48 1.72 ± 1.20 5.52 ± 2.45 5.00 ± 2.26 43
60 The Pact Horror, Mystery, Thriller 4.37 ± 2.00 5.56 ± 2.40 1.86 ± 1.42 6.23 ± 2.05 5.62 ± 2.50 44
accuracy and 0.6916 of mean F1 score. Furthermore, we • Low-High: Valence/Arousal with Clustering
performed repeated measure ANOVA, with Greenhouse- As shown in Table IV, V and A scores appeared to
Geisser correction for statistical analysis. It was found be widely distributed given their considerable standard
that the mean accuracy of low-high A classification was deviation values. It could be the case that larger pop-
significantly different among the application of EEG, E4, ulation is by default associated with higher standard
and Fusion data (F(1.994, 15.953)=15.791), p < 0.05, deviation. Hence, the fixed threshold strategy might not
df(factor, error)=1.994,15.953). Further, the Bonferroni be suitable in this scenario. To address this potential
post hoc test revealed that Fusion (71.574 ± 1.578%) was issue, we tested two clustering methods, K-Means and
significantly better than using either EEG only (67.442 ± GMM. As shown in Table V, K-means clustering with 4
1.644%) or E4 only (62.866 ± 0.253%). clusters was likely the most suitable method as indicated
9
TABLE VIII: Accuracy and F1 scores from different The emotion recognition task in this study resorted to
EEG frequency bands. the measurement of valence and arousal experienced by the
viewers during video presentation. In most studies, ground
Accuracy F1 Score
Frequency Band Valence Arousal Valence Arousal truth labeling relies on an empirically set threshold for high-
θ 0.6718 0.6770 0.6363 0.6518 low partitioning [8]–[10], [12]. However, the list of selected
α 0.7003 0.6434 0.6640 0.6123 titles in Table IV shows large standard deviations for self-
β 0.6925 0.6796 0.6557 0.6590
γ 0.6719 0.6667 0.6286 0.6394 assessment scores. In a large population, simple threshold-
ing might not work well since different people are likely
Optimal values column-wise shown in boldface to experience varying levels of low or high valence and
arousal. Thus, a simple mathematical based algorithm, K-
means clustering, was proposed for labeling the ground truth.
using Fusion (72.350 ± 1.191%) was significantly better As reported in Table VI, the proposed method for ground
than using only E4 (63.567 ± 1.450%). Table VI reports truth labeling may lend support to attaining high classification
the V and A classification results from DREAMER [12], performance, especially in valence classification. Moreover,
DEAP [9], MAHNOB-HCI [8], and DECAF [10] datasets the valence and arousal classification results among using
all using identical feature extraction and classification EEG, peripheral signals, and fusion were notably consistent
method. The observed performance of our data seemed to compared to the conventional ground truth labeling method.
be lending support to the capability of the consumer grade Hence, labeling by K-means clustering might be more suitable
brain sensor, OpenBCI, as a versatile tool in emotion in future emotional studies with a large number of participants.
recognition studies. Moreover, the outcome of emotion recognition from OpenBCI
• EEG Electrode Channels and Frequency Bands was comparable to state-of-the-art works featuring high-end or
The results from channel selection are presented in Ta- expensive EEGs (both in accuracy and F1 score) as can be seen
ble VII with the mean accuracy and mean F1 score from in Table VI, including one involving MEG, a functional brain
nine folds. In the low-high classification task for V, the mapping method (considerably more expensive than EEG).
condition with EEG channels from Fz , Cz , Pz , Oz as Table VII and Table VIII display the study results of EEG
the input features achieved a mean accuracy of 71.84% factors – i.e., electrode channel selection for future devel-
and a mean F1 score of 0.6741. However, for A, the opment of user-friendly devices in emotion recognition, and
mean accuracy reached 63.57% and the mean F1 score frequency selection to identify more effective EEG frequency
reached 0.6258 using T3 , T4 , Oz . Finally, the results bands as classification features. Referring to the tables, T3 , T4 ,
obtained from varying EEG frequency bands, as shown Oz achieved the best results for arousal classification both in
in Table VIII, indicate the mean accuracy and mean F1 terms of accuracy and F1 score, while the middle line (Fz , Cz ,
score from nine folds. The classification accuracy for V Pz , Oz ) was promising for further improvement of an emotion
reached 70.03% by using EEG features from α band and recognition algorithm in valence for low-high classification.
A reached 67.96% by using EEG features from β band. Taking both results together, T3 , T4 , Oz , Fz , Cz , Pz with
In terms of F1 score, the accuracy results were similar. all EEG frequency bands as input features offer the best path
The features from α bands provided the best results in V for developing practical, usable consumer grade devices for
and the features from β band provided the best results in mental state recognition, especially one concerning valence
A (0.6640 and 0.6590, respectively). We then carried out and arousal. True to any scientific investigation, there are
repeated measures ANOVA, however, there appeared to limitations inherent to this study, one of which comes down to
be no significant difference between the selection results ICA implementation. We were able to pinpoint the axes with
of frequency and channel. EMG traits, however, decided not to remove any of them as
this may lead to a potential loss of important information.
V. D ISCUSSION To our knowledge, this study is the first to carry out an
evaluation of OpenBCI applicability in the domain of emotion
The practice of evoking emotions on command is common- recognition. In comparison to medical grade EEG amplifiers
place in emotion research and lately has gained a foothold in with a greater number of electrodes and sampling frequencies,
extended applications such as classification of affective states OpenBCI demonstrably could hold its own. A consumer grade,
from correlated EEG and peripheral physiological signals. A open-source device has a potential to be a real game changer
number of previous datasets have been generated using audio- for programmers or researchers on the quest for better emotion
visual stimuli, more specifically video clips, to invoke desired recognition tools. The device may facilitate further progress
emotions in individuals while at the same time record their toward online applications since it is inexpensive and possibly
EEG and bodily responses. In the present study, we propose a affordable even to those with more limited purchasing power.
novel method for affective stimulus selection, involving video In the same vein, emotion recognition using peripheral phys-
ratings by 200 participants aged 15–22 and K-means clustering iological data from a real-time bracelet sensor or E4 remains
method in video ranking. A complete list of movie titles used a challenge. E4 is an easy-to-use wearable device resembling
in our study is provided in Table IV. These videos can readily a watch which may be useful as a long-term affective state
be found on YouTube and online websites, should there be a or mental health monitoring system. In fact, a research team
need for further use. recently proposed the algorithm K-nearest neighbors, based on
11
dynamic time warping, to process physiological signals and [10] M. K. Abadi, R. Subramanian, S. M. Kia, P. Avesani, I. Patras, and
perform affective computing on E4-derived datasets [72]. We N. Sebe, “Decaf: Meg-based multimodal database for decoding affective
physiological responses,” IEEE Transactions on Affective Computing,
might also be able to adapt this work for OpenBCI. Besides, vol. 6, no. 3, pp. 209–222, 2015.
we have two ongoing projects on deep learning and EEG [11] W.-L. Zheng and B.-L. Lu, “Investigating critical frequency bands and
in which the data from this study could be incorporated for channels for eeg-based emotion recognition with deep neural networks,”
IEEE Transactions on Autonomous Mental Development, vol. 7, no. 3,
further investigation [73], [74]. pp. 162–175, 2015.
[12] S. Katsigiannis and N. Ramzan, “Dreamer: a database for emotion
VI. C ONCLUSION recognition through eeg and ecg signals from wireless low-cost off-
the-shelf devices,” IEEE journal of biomedical and health informatics,
We present evidence in support of the applicability of vol. 22, no. 1, pp. 98–107, 2018.
OpenBCI, an open-source, consumer grade EEG device for [13] U. Rashid, I. Niazi, N. Signal, and D. Taylor, “An eeg experimental study
evaluating the performance of texas instruments ads1299,” Sensors,
emotion recognition research. A brief summary of our effort vol. 18, no. 11, p. 3721, 2018.
to collect this evidence is as follows. EEG and peripheral [14] M. Duvinage, T. Castermans, M. Petieau, T. Hoellinger, G. Cheron,
physiological signals were collected from 43 participants as and T. Dutoit, “Performance of the emotiv epoc headset for p300-based
applications,” Biomedical engineering online, vol. 12, no. 1, p. 56, 2013.
they were being shown a sequence of short movie trailers [15] N. A. Badcock, P. Mousikou, Y. Mahajan, P. De Lissa, J. Thie, and
expected to invoke distinctive emotions. The participants were G. McArthur, “Validation of the emotiv epoc® eeg gaming system for
prompted to score their experienced valence and arousal for measuring research quality auditory erps,” PeerJ, vol. 1, p. e38, 2013.
[16] E. Ratti, S. Waninger, C. Berka, G. Ruffini, and A. Verma, “Comparison
each video. Subsequently, we applied K-means clustering of medical and consumer wireless eeg systems for use in clinical trials,”
algorithm on these valence and arousal ratings in order to Frontiers in human neuroscience, vol. 11, p. 398, 2017.
establish ground truth labels for low and high cluster, dissim- [17] O. E. Krigolson, C. C. Williams, A. Norton, C. D. Hassall, and F. L.
Colino, “Choosing muse: Validation of a low-cost, portable eeg system
ilar to reported studies that performed labeling by empirical for erp research,” Frontiers in neuroscience, vol. 11, p. 109, 2017.
thresholding. We found that our prediction outcome appeared [18] https://www.emotiv.com/.
in a similar performance range to those derived from state- [19] https://store.myndplay.com/.
[20] http://neurosky.com/.
of-the-art datasets that were acquired by medical grade EEG [21] https://choosemuse.com/.
systems. The ultimate goal of this study is to inform and [22] http://openbci.com/.
inspire researchers and engineers alike on the practicality of [23] https://www.ant-neuro.com/.
[24] https://compumedicsneuroscan.com/.
OpenBCI as a recording device in the development of online, [25] http://www.gtec.at/.
emotional EEG-related applications. Our experimental data are [26] https://www.advancedbrainmonitoring.com/.
available to academic peers and researchers upon request. [27] https://www.neuroelectrics.com/.
[28] https://www.brainproducts.com/.
[29] J. A. M. Correa, M. K. Abadi, N. Sebe, and I. Patras, “Amigos: A
ACKNOWLEDGMENT dataset for affect, personality and mood research on individuals and
groups,” IEEE Transactions on Affective Computing, 2018.
The authors would like to thank Sombat Ketrat for his sup- [30] https://www.empatica.com/research/e4/.
port and suggestions on server installation for data storage and [31] https://www.tobiipro.com/.
computing. Our thank also extends to Irawadee Thawornbut [32] J.-M. Lpez-Gil, J. Virgili-Gom, R. Gil, T. Guilera, I. Batalla, J. Soler-
Gonzlez, and R. Garca, “Method for improving eeg based emotion
for her assistance in the setting up of the preliminary study. recognition by combining it with synchronized biometric and eye
tracking technologies in a non-invasive and low cost way,” Frontiers
R EFERENCES in Computational Neuroscience, vol. 10, p. 85, 2016.
[33] B. Nakisa, M. N. Rastgoo, D. Tjondronegoro, and V. Chandran,
[1] J. A. Russell, “A circumplex model of affect.” Journal of personality “Evolutionary computation algorithms for feature selection of eeg-
and social psychology, vol. 39, no. 6, p. 1161, 1980. based emotion recognition using mobile sensors,” Expert Systems with
[2] R. W. Picard, Affective computing. MIT press, 2000. Applications, vol. 93, pp. 143 – 155, 2018.
[3] P. Reis, F. Hebenstreit, F. Gabsteiger, V. von Tscharner, and [34] J. Kim, J. Seo, and T. H. Laine, “Detecting boredom from eye gaze and
M. Lochmann, “Methodological aspects of eeg and body dynamics eeg,” Biomedical Signal Processing and Control, vol. 46, pp. 302 – 313,
measurements during motion,” Frontiers in human neuroscience, vol. 8, 2018.
p. 156, 2014. [35] K. Dhindsa and S. Becker, “Emotional reaction recognition from eeg,” in
[4] P. C. Petrantonakis and L. J. Hadjileontiadis, “Emotion recognition from 2017 International Workshop on Pattern Recognition in Neuroimaging
eeg using higher order crossings,” IEEE Transactions on information (PRNI), June 2017, pp. 1–4.
Technology in Biomedicine, vol. 14, no. 2, pp. 186–197, 2009. [36] Y. Liu, M. Yu, G. Zhao, J. Song, Y. Ge, and Y. Shi, “Real-time
[5] Y.-P. Lin, C.-H. Wang, T.-P. Jung, T.-L. Wu, S.-K. Jeng, J.-R. Duann, and movie-induced discrete emotion recognition from eeg signals,” IEEE
J.-H. Chen, “Eeg-based emotion recognition in music listening,” IEEE Transactions on Affective Computing, vol. 9, no. 4, pp. 550–562, Oct
Transactions on Biomedical Engineering, vol. 57, no. 7, pp. 1798–1806, 2018.
2010. [37] H. Shahabi and S. Moghimi, “Toward automatic detection of brain
[6] D. Nie, X.-W. Wang, L.-C. Shi, and B.-L. Lu, “Eeg-based emotion recog- responses to emotional music through analysis of eeg effective connec-
nition during watching movies,” in 2011 5th International IEEE/EMBS tivity,” Computers in Human Behavior, vol. 58, pp. 231 – 239, 2016.
Conference on Neural Engineering. IEEE, 2011, pp. 667–670. [38] A. Martnez-Rodrigo, A. Fernndez-Sotos, J. M. Latorre, J. Moncho-
[7] C. A. Kothe, S. Makeig, and J. A. Onton, “Emotion recognition from Bogani, and A. Fernndez-Caballero, “Neural correlates of phrase
eeg during self-paced emotional imagery,” in 2013 Humaine Association rhythm: An eeg study of bipartite vs. rondo sonata form,” Frontiers
Conference on Affective Computing and Intelligent Interaction, Sep. in Neuroinformatics, vol. 11, p. 29, 2017.
2013, pp. 855–858. [39] A. Fernndez-Soto, A. Martnez-Rodrigo, J. Moncho-Bogani, J. M.
[8] M. Soleymani, J. Lichtenauer, T. Pun, and M. Pantic, “A multimodal Latorre, and A. Fernndez-Caballero, “Neural correlates of phrase
database for affect recognition and implicit tagging,” IEEE Transactions quadrature perception in harmonic rhythm: An eeg study using a
on Affective Computing, vol. 3, no. 1, pp. 42–55, Jan 2012. brain?computer interface,” International Journal of Neural Systems,
[9] S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, vol. 28, no. 05, p. 1750054, 2018, pMID: 29298521.
T. Pun, A. Nijholt, and I. Patras, “Deap: A database for emotion analysis; [40] R. Ramirez, J. Planas, N. Escude, J. Mercade, and C. Farriols, “Eeg-
using physiological signals,” IEEE Transactions on Affective Computing, based analysis of the emotional effect of music therapy on palliative
vol. 3, no. 1, pp. 18–31, 2012. care cancer patients,” Frontiers in Psychology, vol. 9, p. 254, 2018.
12
[41] P. Kumar, R. Saini, P. P. Roy, P. K. Sahu, and D. P. Dogra, “Envi- [66] “The e4 wristband is a wearable research device that offers real-time
sioned speech recognition using eeg sensors,” Personal and Ubiquitous physiological data acquisition and software for in-depth analysis and
Computing, vol. 22, no. 1, pp. 185–199, 2018. visualization.” https://www.empatica.com/en-eu/research/e4/.
[42] M.-H. Lin, S. N. Cross, W. J. Jones, and T. L. Childers, “Applying eeg [67] A. Greco, G. Valenza, L. Citi, and E. P. Scilingo, “Arousal and valence
in consumer neuroscience,” European Journal of Marketing, vol. 52, no. recognition of affective sounds based on electrodermal activity,” IEEE
1/2, pp. 66–91, 2018. Sensors Journal, vol. 17, no. 3, pp. 716–725, 2017.
[43] M. Yadava, P. Kumar, R. Saini, P. P. Roy, and D. P. Dogra, “Analysis [68] “Artifact correction with ica,” https://martinos.org/mne/stable/
of eeg signals and its application to neuromarketing,” Multimedia Tools auto tutorials/preprocessing/plot artifacts correction ica.html#
and Applications, vol. 76, no. 18, pp. 19 087–19 111, 2017. sphx-glr-auto-tutorials-preprocessing-plot-artifacts-correction-ica-py,
[44] A. J. Casson and E. V. Trimble, “Enabling free movement eeg tasks accessed: 2019-05-22.
by eye fixation and gyroscope motion correction: Eeg effects of color [69] A. Gramfort, M. Luessi, E. Larson, D. Engemann, D. Strohmeier,
priming in dress shopping,” IEEE Access, vol. 6, pp. 62 975–62 987, C. Brodbeck, R. Goj, M. Jas, T. Brooks, L. Parkkonen, and M. Hmlinen,
2018. “Meg and eeg data analysis with mne-python,” Frontiers in Neuro-
[45] I. Arapakis, M. Barreda-Angeles, and A. Pereda-Banos, “Interest as a science, vol. 7, p. 267, 2013.
proxy of engagement in news reading: Spectral and entropy analyses of [70] E. Jones, T. Oliphant, P. Peterson et al., “SciPy: Open source scientific
eeg activity patterns,” IEEE Transactions on Affective Computing, pp. tools for Python,” 2001–, [Online; accessed 2019-05-22]. [Online].
1–1, 2018. Available: http://www.scipy.org/
[46] N. Jadhav, R. Manthalkar, and Y. Joshi, “Effect of meditation on [71] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion,
emotional response: An eeg-based study,” Biomedical Signal Processing O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vander-
and Control, vol. 34, pp. 101–113, 2017. plas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duch-
[47] A. T. Poulsen, S. Kamronn, J. Dmochowski, L. C. Parra, and L. K. esnay, “Scikit-learn: Machine learning in Python,” Journal of Machine
Hansen, “Eeg in the classroom: Synchronised neural recordings during Learning Research, vol. 12, pp. 2825–2830, 2011.
video presentation,” Scientific Reports, vol. 7, p. 43916, 2017. [72] A. Albraikan, D. P. Tobón, and A. El Saddik, “Toward user-independent
[48] P. Zioga, F. Pollick, M. Ma, P. Chapman, and K. Stefanov, “enheduannaa emotion recognition using physiological signals,” IEEE Sensors Journal,
manifesto of falling live brain-computer cinema performance: Performer 2018.
and audience participation, cognition and emotional engagement using [73] T. Wilaiprasitporn, A. Ditthapron, K. Matchaparn, T. Tongbuasirilai,
multi-brain bci interaction,” Frontiers in neuroscience, vol. 12, p. 191, N. Banluesombatkul, and E. Chuangsuwanich, “Affective eeg-based per-
2018. son identification using the deep learning approach,” IEEE Transactions
[49] Y. Dai, X. Wang, P. Zhang, and W. Zhang, “Wearable biosensor on Cognitive and Developmental Systems, pp. 1–1, 2019.
network enabled multimodal daily-life emotion recognition employing [74] A. Ditthapron, N. Banluesombatkul, S. Ketrat, E. Chuangsuwanich,
reputation-driven imbalanced fuzzy classification,” Measurement, vol. and T. Wilaiprasitporn, “Universal joint feature extraction for p300 eeg
109, pp. 408–424, 2017. classification using multi-task autoencoder,” IEEE Access, vol. 7, pp.
[50] H. Cai, X. Zhang, Y. Zhang, Z. Wang, and B. Hu, “A case-based 68 415–68 428, 2019.
reasoning model for depression based on three-electrode eeg data,” IEEE
Transactions on Affective Computing, 2018.
[51] H. Jebelli, S. Hwang, and S. Lee, “Eeg-based workers’ stress recognition
at construction sites,” Automation in Construction, vol. 93, pp. 315–324,
2018.
[52] X. Shan, E.-H. Yang, J. Zhou, and V. W.-C. Chang, “Human-building
interaction under various indoor temperatures through neural-signal
electroencephalogram (eeg) methods,” Building and Environment, vol.
129, pp. 46–53, 2018.
[53] J. Rosenthal and G. Benabdallah, “Ibpoet: an interactive & biosensitive
poetry composition device,” in Proceedings of the 2017 ACM Interna-
tional Joint Conference on Pervasive and Ubiquitous Computing and
Proceedings of the 2017 ACM International Symposium on Wearable
Computers. ACM, 2017, pp. 281–284.
[54] R. Munoz, R. Villarroel, T. S. Barcelos, A. Souza, E. Merino, R. Guiñez,
and L. A. Silva, “Development of a software that supports multimodal
learning analytics: A case study on oral presentations,” Journal of
Universal Computer Science, vol. 24, no. 2, pp. 149–170, 2018.
[55] M. Mohammadpour, S. M. R. Hashemi, and N. Houshmand, “Classi-
fication of eeg-based emotion for bci applications,” in 2017 Artificial
Intelligence and Robotics (IRANOPEN), April 2017, pp. 127–131.
[56] M. Mohammadpour, M. M. AlyanNezhadi, S. M. R. Hashemi, and
Z. Amiri, “Music emotion recognition based on wigner-ville distribution
feature extraction,” in 2017 IEEE 4th International Conference on
Knowledge-Based Engineering and Innovation (KBEI), Dec 2017, pp.
0012–0016.
[57] M. A. Hafeez, S. Shakil, and S. Jangsher, “Stress effects on exam
performance using eeg,” in 2018 14th International Conference on
Emerging Technologies (ICET), Nov 2018, pp. 1–4.
[58] L. Rahman and K. Oyama, “Long-term monitoring of nirs and eeg
signals for assessment of daily changes in emotional valence,” in 2018
IEEE International Conference on Cognitive Computing (ICCC), July
2018, pp. 118–121.
[59] R. Munoz, R. Olivares, C. Taramasco, R. Villarroel, R. Soto, M. F.
Alonso-Sánchez, E. Merino, and V. H. C. de Albuquerque, “A new
eeg software that supports emotion recognition by using an autonomous
approach,” Neural Computing and Applications, pp. 1–17.
[60] B. Chris and L. Jennifer, “Frozen,” USA: Walt Disney Pictures, 2013.
[61] M. Bay, “Transformers: The last knight,” USA: Paramount Pictures,
2017.
[62] J. Collet-Serra, “Orphan,” USA: Warner Bros. Pictures, 2009.
[63] D. Chazelle, “La la land,” USA: Lionsgate, 2016.
[64] N. Meyers, “The intern,” USA: Warner Bros. Pictures, 2015.
[65] D. Villeneuve, “Arrival,” USA: Paramount Pictures, 2016.