Deep Learning For Healthcare Applications Based On Physiological Signals: A Review
Deep Learning For Healthcare Applications Based On Physiological Signals: A Review
Deep Learning For Healthcare Applications Based On Physiological Signals: A Review
a review
Oliver Fausta,∗, Yuki Hagiwarab , Tan Jen Hongb , Oh Shu Lihb , U Rajendra Acharyab,c,d
a
Department of Engineering and Mathematics, Sheffield Hallam University, United Kingdom
b
Department of Electronic & Computer Engineering, Ngee Ann Polytechnic, Singapore
c
Department of Biomedical Engineering, School of Science and Technology, SIM University, Singapore
d
Department of Biomedical Imaging, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
Abstract
Background and Objective: We have cast the net into the ocean of knowledge to retrieve the
latest scientific research on deep learning methods for physiological signals. We found 53 research
papers on this topic, published from 01.01.2008 to 31.12.2017.
Methods: An initial bibliometric analysis shows that the reviewed papers focused on Electromyogram
(EMG), Electroencephalogram (EEG), Electrocardiogram (ECG), and Electrooculogram (EOG).
These four categories were used to structure the subsequent content review.
Results: During the content review, we understood that deep learning performs better for big
and varied datasets than classic analysis and machine classification methods. Deep learning
algorithms try to develop the model by using all the available input.
Conclusions: This review paper depicts the application of various deep learning algorithms used
till recently, but in future it will be used for more healthcare areas to improve the quality of
diagnosis.
Keywords: Deep learning, Physiological signals, Electrocardiogram, Electroencephalogram,
Electromyogram, Electrooculogram
1. Introduction
Physiological signals are an invaluable data source which assists in disease detection, reha-
bilitation, and treatment [1]. The signals come from sensors implanted or placed on the skin
[2]. The application areas dictate where the sensors are to be placed [3, 4]. In turn, the sensor
location determines the characteristics of the physiological signal. Relevant information must
be extracted from the physiological signal in order to support a specific healthcare application
[5, 6]. It is difficult to establish what constitutes relevant information, because various medical
ontologies can be used for data interpretation [7]. It is unavoidable that these ontologies con-
tain conflicts and inconsistencies [8]. Another fundamental problem is that physiological signal
interpretation suffers from intra-individual variability [9]. For example, both electrode position
and noise influence the signal waveform [10]. Therefore, human interpretation requires years of
training to acquire specialized knowledge. Even with expert knowledge, manual physiological
signal interpretation suffers from intra- and inter-operator variability [11]. Furthermore, physi-
ological signal interpretation is a tiresome process where human errors can be caused by fatigue
[12, 13]. Computer supported signal analysis is not affected by fatigue related mistakes and it
can also eliminate both intra- as well as inter-observer variability. In addition, most computer
based analysis interpretation can be done quicker and more cost effective when compared to
∗
Corresponding author
Feature extraction
and Feature extraction
assessment
Figure 1: Blcokdiagram for the design of a traditional decision support system based on physiological signals.
Current decision-making systems under-perform for large and varied datasets, because they
fail to process information that is sufficiently diverse to cover all scenarios. More depth is re-
quired to represent all the information in a dataset. Fundamentally, we require implicit knowl-
edge to make good decisions based on the information extracted from physiological signals. In
the past, that implicit knowledge was exclusively found in highly skilled medical practitioners.
Deep learning was designed to overcome these shortcomings by taking into consideration all in-
formation a training dataset has to offer. The method promises to establish implicit knowledge
in a machine. In this review, we investigate the validity of this claim by analysing the ways in
2
which deep learning has been applied to physiological signals. We found promising research in
the areas of Electrocardiogram (ECG), Electroencephalogram (EEG), Electromyogram (EMG)
and Electrooculogram (EOG) based healthcare applications. However, the amount of research
work does not reflect the diversity of healthcare application that can benefit from physiological
signal interpretation. Therefore, we adopt the position that the diversification of the research
is needed. The depth and specialization must come from training the deep learning algorithms
with larger and more varied data sets.
To support our claim that deep learning is a major step towards understanding physiological
signals, we have organized the remainder of the paper as follows. The next section establishes
the importance of implicit knowledge for physiological signal interpretation and it provides
background on deep learning. Section 3 presents bibliometric and content analysis results.
Based on these results, we put forward a range of research gaps in the discussion section. In this
section, we have also discussed limitations and future work. The paper concludes with Section
5.
2. Background
Physiological signals reflect the electrical activity of a specific body part [6]. As such, the
electrical activity provides information about the physiological condition. Traditionally, this
information can be used by medical practitioners for decision making [32, 33]. These decisions
have far reaching consequences on diagnosis, treatment monitoring, drug efficacy tests, and
quality of life. From a machine learning perspective, a medical practitioner is an intelligent
agent who makes good decisions. In order to reach these decisions some sort of knowledge is
required. This knowledge shapes the process which generates actions from both state and input.
For example, a clinician analyses an ECG signal. As that analysis process continues, there is a
growing suspicion that the signal was taken from a patient with coronary artery disease. More
and more suspicious waveform segments emerge until a positive diagnosis is reached. From a
processing perspective, the ECG signal is the input, the reading clinician is the system and the
system state is reflected in the growing suspicion.
In its simplest form, knowledge can be explicitly expressed as mathematical rules and physical
facts. Engineers make use of this explicit knowledge to create expert systems [34]. However, these
systems are limited to a specific domain and they operate in highly controlled environments.
Only for these environments, we are reasonably sure about the physical facts and mathematical
rules which shape the correct behaviour. Furthermore, expert systems don’t reflect the concept
of suspicion and common sense. Overcoming these limitations requires implicit knowledge which
can be found in the complex wiring and the synaptic setup of biological brains as well as the
mechanical and sensory properties of biological bodies. The only way for computers to mimic
implicit knowledge is to learn from examples. The idea is to find a way to learn general features
in order to make sense of new data. This description highlights the central role of data for
establishing implicit knowledge. The amount of data must be sufficiently large to provide many
training examples from which a large set of parameters can be extracted. Only a large number
of parameters give rise to the richness of class functions which model the implicit knowledge.
Deep learning is a change in basic assumptions of artificial intelligence algorithm design [35].
This change percolates through to all application areas of machine learning, such as computer
vision, speech recognition, natural language processing and indeed diagnosis support [36]. Cen-
tral to deep learning are the ideas of parallel processing and networked entities. The next section
details these ideas by discussing the deep learning methods.
3
each successive hidden layer of neurons [31]. The term “deep” is derived from the numerous
hidden layers in the Artificial Neural Network (ANN) structure.
The ANN algorithm models the functionality of a biological brain [37]. The model is real-
ized with a structure that is made up of input-, hidden-, and output-layers, as shown in Figure
2. Every neuron or node (nerve cell) is connected to each neuron in the next layer through a
connection link. A nerve cell is made up of axon (output), dendrites (input), a node (soma), nu-
cleus (activation function), and synapses (weights) [38]. The activation function in the artificial
neuron acts as the nucleus in a biological neuron whereas the input signals and its respective
weights model the dendrites and synapses respectively. Figure 3 illustrates the neuron structure.
Unfortunately, the ANN structure is receptive to translation and shift deviation, which
may adversely affect the classification performance [39]. To eliminate these shortcomings, an
extended version of ANN, the Convolutional Neural Network (CNN), was developed [40]. The
CNN architecture ensures translation and shift invariance [41]. Figure 4 illustrates a generic
CNN network structure. It is a feed-forward network, which comprises of: convolution, pooling,
and fully-connected, layers [41]. They are briefly explained below.
1. Convolution layer: The input sample is convolved with a kernel (weights) in this layer. The
output of this layer is the feature map. The stride controls how much the kernel convolves
with the input sample. The convolution operation acts as a feature extractor by learning
from the diverse input signals. The extracted features can be used for classification in
subsequent layers. Figure 5 illustrates a convolution operation between f (input) and g
(kernel), giving an output c. Equations 1 and 2 provide an example of the convolution
4
Figure 3: Neuron structure
calculation.
c(1) = 1 × 3 + 2 × 4 + 3 × 1 = 14
c(2) = 2 × 3 + 3 × 4 + 4 × 1 = 22 (1)
c(3) = 3 × 3 + 4 × 4 + 5 × 1 = 30
Output1 = 22 × W1 + 38 × W3 + 54 × W5
(2)
Output2 = 22 × W2 + 38 × W4 + 54 × W6
2. Pooling layer: This is a down-sampling layer where the pooling operation is employed to
reduce the spatial dimension of the input sample, while retaining the significant informa-
tion. The pooling operation can be average, max, or sum. The max-pooling operation is
typically employed. An example of a max-pooling operation, with a stride of 2, is shown
Figure 6. In this example, the number of input samples are halved by retaining only the
maximum value within a selected stride. In a stride that contains 14 and 22, the value 22
is retained, and, 14 is discarded.
3. Fully-connected layer: Fully-connected signifies that each neuron in the previous layer is
connected to all the neurons in the current layer. The total number of fully-connected
neurons in the final layer determines the number of classes. Figure 7 provides a graphical
representation of a fully-connected layer. The neurons are all connected and each connec-
tion has a specific weight. This layer establishes a weighted sum of all the outputs from
the previous layer to determine a specific target output.
A leaky rectifier linear unit [42] is used as an activation function after the convolution layer.
The purpose is to map the output to the input set and introduce non-linearity as well as sparsity
to the network. The CNN is trained with backpropagation [43] and the hyperparameters may
be tuned for optimal training performance.
In addition to CNN, there are other deep learning architectures, such as autoencoder [44],
deep generative models [45] [46] and Recurrent Neural Network (RNN), that can be used to
monitor the physiological signals [47].
5
Figure 4: CNN network structure
The autoencoder is an unsupervised neural network that is trained to imitate the input
to its output. The dimension of the input is the same as the dimension of the output. The
encoders are stacked together to form a deep autoencoder network. The autoencoder takes
unlabelled inputs, encodes these inputs, and subsequently it reconstructs the inputs as precisely
as possible. Hence, the network structure must determine which data features are significant.
An autoencoder consists of three layers: input, output, and hidden. Encoding and decoding
are the two main steps in the autoencoder algorithm. During encoding and decoding, the same
weights are employed to encode the feature and reconstruct an output sample in the output.
Autoencoders are trained with a backpropagation algorithm that employs a metric known as the
loss function [43]. That function computes the amount of information which is lost during input
reconstruction. Thus, a network structure with a small loss value will produce a reconstruction
that is nearly identical to the input sample [37].
Deep Belief Network (DBN) [45] and Restricted Boltzmann Machine (RBM) [48] are common
forms of the deep generative model, where DBNs are stacked layers of RBM. The RBM is made
up of a two-layer neural net with one visible and one hidden layer. Each node in the layer learns
a single feature from the input data by making random decisions on whether to transmit the
input or not. The nodes in the first (visible) layer are connected to every node in the second
(hidden) layer. The RBM takes the inputs and translates them into a set of numbers that encode
6
Figure 6: Max pooling layer
the inputs. Then, a backward translation is implemented to reconstruct the inputs. The RBM
network is trained to reconstruct through forward and backward passes [49, Chapter 6].
The DBN is a probabilistic model with several hidden layers [50]. It can be efficiently
trained by using a technique known as greedy layer-wise pre-training [51]. Typically, the DBNs
are stacked layers of RBM. The first layer of the RBM is trained to reconstruct its input. After
which, the first hidden layer is considered as the visible layer and it is trained using the outputs
from the input layer. This process is iterated until all layers in the structure are trained [45].
In contrast to the feed-forward network, the RNN employs a recursive approach (recurrent
network) whereby the network performs a routine task with the output being dependent on the
previous computation. This functionality is created with inbuilt memory. The most common
type of RNN is the Long Short-Term Memory (LSTM) network [52]. It has the capability to
learn long-term dependencies. The LSTM algorithm incorporates a memory block with three
gates: the input, output, and forget gate. These gates control the cell state and decide which
information to add or remove from the network. This process repeats for every input.
1. Input gate: decides what new information is to be stored and updated in the cell state.
2. Output gate: judges what information is used based on the cell state.
3. Forget gate: evaluates what information is redundant and discards it from the cell state.
These deep learning architectures have demonstrated their potential by surpassing the perfor-
mance of traditional machine learning techniques [31]. Furthermore, deep learning algorithms
minimize the need for feature engineering. The next section, focuses on scientific work that
applied deep learning to physiological signal interpretation for health care applications.
3. Review
In this review, we consider 53 articles focusing on deep learning methods applied to phys-
iological signals for healthcare applications. These papers were published in the period from
01.01.2008 to 31.12.2017. Figure 8 shows the yearly distribution of 53 articles together with a
trend line [53]. Apart from the data distribution, the figure also features a trend line which was
7
generated with a linear regression approach [54]. The large positive gradient of the trend line
indicates more papers, on this topic, published in recent years.
30
30 Trend line gradient: 2.46
25
Number of articles
20
15
10 9 8
5 2 2
0 0 0 1 1
0
2008 2010 2012 2014 2016
Year
Figure 8: Number of articles on deep learning, published each year from 01.01.2008 to 31.12.2017
Our review of the 53 articles is based on two analysis methods. The first method is a
bibliometric analysis, which helps us to detect the structure that governs relationship between
the paper topics. We use this information to arrange the second analysis method, which deals
with the content of the reviewed papers. The paper content is dissected in terms of common
parameters that enable us to compare the studies across different healthcare applications.
1
Web page (Last accessed 11.12.2017): http://www.vosviewer.com/
2
In terms of the number of papers that include a keyword that was mapped to the cluster topics.
8
Figure 9: Network visualization for the author supplied keywords
Table 1: Topic cluster summary, where C is the Cluster number. Colour indicates the cluster colour as shown in
Figure 9.
C Topics Colour
1 emg, cnn, cognitive states, dnn, fnirs, hand gesture, infrared spec- Red
troscopy, ml, myoelectric interfaces, prostheses, real-time, recognition,
rehabilitation robotics, speech, wearable, word generation
2 eeg, bci, learning, motor imagery, multifractal attribute, neuromorphic Blue
computing, noise, wavelet leader
3 ecg, aed, arrhythmia, contaminant mitigation, cvd, motion artifact Yellow
4 eog, dbn, heartbeat, osa, rnn, signal, sleep, svm Green
9
Table 2: Summary of works carried out using DL algorithms
with EMG signals.
Wand and Schmidhu- Speech recognition Deep Neu- EMG-UKA Corpus [68] DNN outperforms a Gaussian
ber, 2016 [67] ral Network Mixture Model (GMM) front
(DNN) end
Allard et al., 2016 [69] Robotic arm guidance CNN 18 subjects performing accuracy: ≈ 97.9%
7 gestures
Wand and Schultz, 2014 Speech recognition DNN 25 sessions from 25 sub- Visualization of hidden nodes
[70] jects activity
3.2.1. Deep learning methods applied to EMG
The EMG measures the electrical activity of skeletal muscles [57]. The electrical sensors,
known as electrodes, are placed on the skin above the muscles of interest [58]. The EMG signals
indicate: muscle activation, force produced by the muscle, and muscle state [59]. All these factors
are superimposed, i.e. the EMG signal measurement picks up the contributions from different
sources and sums them up. Hence, it is difficult to gain information about one particular aspect
by observing the EMG signal. Table 2 details the research work which describe the deep learning
methods used to analyze the EMG signal. Individual columns healthcare application area, Deep
Learning (DL) algorithm, the data used for the study, and the study results. As such, the DL
algorithms were introduced in Section 2.1.
11
Table 3: Summary of works carried out using DL algorithms
with EEG signals.
Ren and Wu, 2014 [124] Feature extraction CNN Dataset III in 2003 BCIC II, Accuracy: 87.33%
Dataset Iva in 2005 BCIC III,
Dataset III in 2005 BCIC III
[110, 111, 112]
Ahmed et al., 2013 [125] Detecting target images DBN Not specified DBN outperforms SVM.
Mirowski et al., 2008 Epileptic seizure predic- CNN Freiburg EEG DB [104, 105, zero-false-alarm seizure pre-
[126] tion 106, 107, 108] diction on 20 patients out of
21
Cecotti and Graeser, Motor imagery classifi- CNN 2 subjects 5 trails of about 3 Recognition accuracy: 53.47%
2008 [127] cation minutes
Table 4: Summary of works carried out using DL algorithms
with ECG signals.
Taji et al., 2017 [145] Signal quality classifica- RBM MIT-BIH arrhythmia DB accuracy: 99.5%
tion [129]
Lou et al., 2017 [138] Heartbeat classification RBM MIT-BIH arrhythmia DB RBM-based model to extract
[135] representative features and to
reduce the data dimensional-
ity
Muduli et al., 2017 [146] Fetal-ECG signal recon- Stacked De- Abdominal non-invasive accuracy: 99.5%
struction noising Au- FECG DB [129]
toencoder
Kiranyaz et al., 2016 Heart beat classification CNN MIT/BIH arrhythmia DB accuracy: 99%
[147] [129]
Zheng et al., 2014 [148] Congestive heart failure CNN BIDMC Congestive Heart accuracy: 94.67%
detection Failure data set [129]
Table 5: Summary of works carried out using DL algorithms
with EOG and other signals.
Apart from the algorithms, we have also studied the type of data used to conduct these
studies. Table 7 indicates a summary of the data type used to implement the DL algorithm.
Row one indicates that 28 studies were conducted using the public databases. Only 20 studies
used private databases and 3 studies have used both databases. 16 out of 17 studies using ECG
and 8 out of 23 studies on EEG have used public data. Indeed, 12 studies have used private
EEG data to implement the deep learning algorithms.
Conventional machine learning approaches require lots of time and effort for feature selection.
These features must extract relevant information from huge and diverse data in order to produce
the best diagnostic performance. Furthermore, the best algorithm is unknown and thus, a lot
of trial and error is necessary to select the best feature extraction algorithms and classification
methods to develop a robust and reliable decision support systems for the physiological signals.
17
On the other hand, deep learning eliminates the need for feature extraction and feature selection.
The decision-making algorithm can consider all available evidence [155].
The training algorithms for deep leaning systems have a high computational complexity.
Hence, this results in a high run-time complexity which translates into a long training time
[156, 157, 158]. This will be a problem during the design phase, because during this phase a
designer must decide what deep leaning architecture to use. Once the architecture is chosen, the
tuning parameters must be adjusted. Both the structure selection and parameter adjustment
will basically influence the model. Hence, it is necessary to have many test runs. Shortening the
training phase of deep leaning models is an active area of research [159]. The challenge is speeding
up the training process in a parallel distributed processing system [160]. The network between
the individual processors becomes the bottle neck [161]. Graphics Processing Units (GPUs) can
be used to reduce the network latency [162].
For this state of the art technology, the training time may be an issue, because it can influence
the model selection strategies. To be specific, none of the reviewed papers approached the model
selection process with statistical methods, such as cross validation [163]. We can only assume
that the deep learning architectures, used in the reviewed scientific work, were selected based
on single run trial. Similarly, we have to assume that the hyperparameters were also optimized
by training the network once. This is a serious shortcoming, because these statistical validation
methods reduce the sample selection bias [164, 165].
In addition, the use of deep learning can also be extended from one-dimensional physiological
signals to two-dimensional medical images. Researchers are also exploring the benefits utilizing
deep learning in medical image analyses [166]. It was reported that the automated staging and
detection of Alzheimers disease, breast cancer, and lung cancer has shown optimistic diagnostic
performances.
5. Conclusion
In this paper, we reviewed 53 papers on deep learning methods applied to healthcare ap-
plications based on physiological signals. The analysis was carried out in two distinct steps.
The first step was bibliometric keyword analysis based on the co-occurrence map. This analysis
step reveals the connection between the topics covered in the reviewed papers. We found four
distinct clusters, one for each physiological signal. That result helped us to structure the second
analysis step, which focuses on the paper content. As such, the paper content was established
by extracting the specific application area, the deep learning algorithm, system performance,
and the types of dataset used to develop the system.
The fact that deep learning algorithms performs well with large and diverse datasets which
has two consequences. First, the dataset becomes critically important for the system design.
Therefore, we focused our efforts on this criterion during our analysis. We found that, the scien-
tific work, documented in 31 of the reviewed papers, was based on one or more freely available
datasets. Therefore, we predict that the importance of these freely available public datasets may
increase. The other consequence is that deep learning algorithms will perform well in practical
settings, because clinical routine produces lots of data with large variations. However, none
of the reviewed papers verified this in a practical setting. Another fundamental point is that,
our literature survey yielded only 53 papers. The small number of studies imply that there is
scope for future work. To be specific, 53 papers do not reflect the comprehensive healthcare
applications based on the physiological signals. In future, there may be more advanced deep
learning algorithms focused on the early detection of diseases using physiological signals.
Acknowledgements
18
Competing interests: The authors declare that they have no conflict of interest.
Statements of ethical approval: For this review article, the authors did not undertake work
that involved human participants or animals.
Acronyms
References
[2] D. Wang, Y. Shang, Modeling physiological data with deep belief networks, International
journal of information and education technology (IJIET) 3 (2013) 505.
[4] W. Van Drongelen, Signal processing for neuroscientists: an introduction to the analysis
of physiological signals, Elsevier, 2006.
[5] L. Sörnmo, P. Laguna, Bioelectrical signal processing in cardiac and neurological applica-
tions, volume 8, Academic Press, 2005.
[6] S. R. Devasahayam, Signals and systems in biomedical engineering: signal processing and
physiological systems modeling, Springer Science & Business Media, 2012.
[7] R. Miotto, F. Wang, S. Wang, X. Jiang, J. T. Dudley, Deep learning for healthcare:
review, opportunities and challenges, Briefings in Bioinformatics (2017) bbx044.
19
[9] L. K. Morrell, F. Morrell, Evoked potentials and reaction times: a study of intra-individual
variability, Electroencephalography and Clinical Neurophysiology 20 (1966) 567–575.
[11] O. Faust, E. Y. Ng, Computer aided diagnosis for cardiovascular diseases based on ecg
signals: A survey, Journal of Mechanics in Medicine and Biology 16 (2016) 1640001.
[13] T. Vertinsky, B. Forster, Prevalence of eye strain among radiologists: influence of viewing
variables on symptoms, American Journal of Roentgenology 184 (2005) 681–686.
[15] A. Miller, B. Blott, et al., Review of neural network applications in medical imaging and
signal processing, Medical and Biological Engineering and Computing 30 (1992) 449–464.
[16] O. Faust, W. Yu, N. A. Kadri, Computer-based identification of normal and alcoholic eeg
signals using wavelet packets and energy measures, Journal of Mechanics in Medicine and
Biology 13 (2013) 1350033.
[17] O. Faust, U. R. Acharya, T. Tamura, Formal design methods for reliable computer-aided
diagnosis: a review, IEEE reviews in biomedical engineering 5 (2012) 15–28.
[18] K. Y. Zhi, O. Faust, W. Yu, Wavelet based machine learning techniques for electrocar-
diogram signal analysis, Journal of Medical Imaging and Health Informatics 4 (2014)
737–742.
[19] U. R. Acharya, E. C.-P. Chua, O. Faust, T.-C. Lim, L. F. B. Lim, Automated detection
of sleep apnea from electrocardiogram signals using nonlinear parameters, Physiological
measurement 32 (2011) 287.
[21] O. Faust, U. R. Acharya, H. Adeli, A. Adeli, Wavelet-based eeg processing for computer-
aided seizure detection and epilepsy diagnosis, Seizure 26 (2015) 56–64.
[23] R. Rao, R. Derakhshani, A comparison of eeg preprocessing methods using time delay
neural networks, in: Neural Engineering, 2005. Conference Proceedings. 2nd International
IEEE EMBS Conference on, IEEE, pp. 262–264.
[24] T. Kalayci, O. Ozdamar, Wavelet preprocessing for automated neural network detection
of eeg spikes, IEEE engineering in medicine and biology magazine 14 (1995) 160–166.
20
[25] R. Kohavi, G. H. John, Wrappers for feature subset selection, Artificial intelligence 97
(1997) 273–324.
[28] H. Yoon, K. Yang, C. Shahabi, Feature subset selection and feature ranking for mul-
tivariate time series, IEEE transactions on knowledge and data engineering 17 (2005)
1186–1198.
[29] H. Liu, H. Motoda, Computational methods of feature selection, CRC Press, 2007.
[30] Y. Bengio, A. C. Courville, P. Vincent, Unsupervised feature learning and deep learning:
A review and new perspectives, CoRR, abs/1206.5538 1 (2012).
[31] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–444.
[33] O. Faust, L. M. Yi, L. M. Hua, Heart rate variability analysis for different age and gender,
Journal of Medical Imaging and Health Informatics 3 (2013) 395–400.
[34] P. D. McAndrew, D. L. Potash, B. Higgins, J. Wayand, J. Held, Expert system for pro-
viding interactive assistance in solving problems such as health care management, 1996.
US Patent 5,517,405.
[35] M. Längkvist, L. Karlsson, A. Loutfi, A review of unsupervised feature learning and deep
learning for time-series modeling, Pattern Recognition Letters 42 (2014) 11–24.
[36] Y. Bengio, O. Delalleau, On the expressive power of deep architectures, in: Algorithmic
Learning Theory, Springer, pp. 18–36.
[41] Y. LeCun, Y. Bengio, et al., Convolutional networks for images, speech, and time series,
The handbook of brain theory and neural networks 3361 (1995) 1995.
[42] K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human-level per-
formance on imagenet classification, in: Proceedings of the IEEE international conference
on computer vision, pp. 1026–1034.
21
[44] G. E. Hinton, R. R. Salakhutdinov, Reducing the dimensionality of data with neural
networks, science 313 (2006) 504–507.
[45] G. E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets,
Neural computation 18 (2006) 1527–1554.
[47] J. J. Hopfield, Neural networks and physical systems with emergent collective compu-
tational abilities, in: Spin Glass Theory and Beyond: An Introduction to the Replica
Method and Its Applications, World Scientific, 1987, pp. 411–415.
[48] H. Larochelle, M. Mandel, R. Pascanu, Y. Bengio, Learning algorithms for the classification
restricted boltzmann machine, Journal of Machine Learning Research 13 (2012) 643–669.
[55] N. J. van Eck, L. Waltman, Visualizing bibliometric networks, in: Measuring scholarly
impact, Springer, 2014, pp. 285–320.
[56] L. Waltman, N. J. van Eck, E. C. Noyons, A unified approach to mapping and clustering
of bibliometric networks, Journal of Informetrics 4 (2010) 629–635.
[57] J. V. Basmajian, C. De Luca, Muscles alive, Muscles alive: their functions revealed by
electromyography 278 (1985) 126.
[59] C. De Luca, Myoelectric manifestation of localized muscular fatigue in humans, in: Ieee
Transactions on Biomedical Engineering, volume 30, IEEE-INST ELECTRICAL ELEC-
TRONICS ENGINEERS INC 445 HOES LANE, PISCATAWAY, NJ 08855 USA, pp.
531–531.
[60] P. Xia, J. Hu, Y. Peng, Emg-based estimation of limb movement using deep learning with
recurrent convolutional neural networks, Artificial organs (2017).
[61] X. Zhai, B. Jelfs, R. H. Chan, C. Tin, Self-recalibrating surface emg pattern recognition for
neuroprosthesis control based on convolutional neural network, Frontiers in neuroscience
11 (2017) 379.
22
[62] M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A.-G. M. Hager, S. Elsig, G. Giat-
sidis, F. Bassetto, H. Müller, Electromyography data for non-invasive naturally-controlled
robotic hand prostheses, Scientific data 1 (2014) 140053.
[63] K.-H. Park, S.-W. Lee, Movement intention decoding based on deep learning for multiuser
myoelectric interfaces, in: Brain-Computer Interface (BCI), 2016 4th International Winter
Conference on, IEEE, pp. 1–2.
[64] M. Atzori, A. Gijsberts, I. Kuzborskij, S. Elsig, A.-G. M. Hager, O. Deriaz, C. Castellini,
H. Müller, B. Caputo, Characterization of a benchmark database for myoelectric movement
classification, IEEE Transactions on Neural Systems and Rehabilitation Engineering 23
(2015) 73–83.
[65] M. Atzori, M. Cognolato, H. Müller, Deep learning with convolutional neural networks
applied to electromyography data: A resource for the classification of movements for pros-
thetic hands, Frontiers in neurorobotics 10 (2016).
[66] W. Geng, Y. Du, W. Jin, W. Wei, Y. Hu, J. Li, Gesture recognition by instantaneous
surface emg images, Scientific reports 6 (2016) 36571.
[67] M. Wand, J. Schmidhuber, Deep neural network frontend for continuous emg-based speech
recognition., in: INTERSPEECH, pp. 3032–3036.
[68] M. Wand, M. Janke, T. Schultz, The emg-uka corpus for electromyographic speech pro-
cessing, in: Fifteenth Annual Conference of the International Speech Communication
Association, pp. 1593–1597.
[69] U. C. Allard, F. Nougarou, C. L. Fall, P. Giguère, C. Gosselin, F. Laviolette, B. Gosselin,
A convolutional neural network for robotic arm guidance using semg based frequency-
features, in: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Con-
ference on, IEEE, pp. 2464–2470.
[70] M. Wand, T. Schultz, Pattern learning with deep neural networks in emg-based speech
recognition, in: Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual
International Conference of the IEEE, IEEE, pp. 4200–4203.
[71] Q. Liu, K. Chen, Q. Ai, S. Q. Xie, recent development of signal processing algorithms for
ssvep-based brain computer interfaces, Journal of Medical and Biological Engineering 34
(2014) 299–309.
[72] Y.-F. Chen, K. Atal, S. Xie, Q. Liu, A new multivariate empirical mode decomposition
method for improving the performance of ssvep-based brain computer interface, Journal
of Neural Engineering (2017).
[73] T.-P. Jung, S. Makeig, M. Westerfield, J. Townsend, E. Courchesne, T. J. Sejnowski,
Removal of eye activity artifacts from visual event-related potentials in normal and clinical
subjects, Clinical Neurophysiology 111 (2000) 1745–1758.
[74] A. Schlögl, C. Keinrath, D. Zimmermann, R. Scherer, R. Leeb, G. Pfurtscheller, A fully
automated correction method of eog artifacts in eeg recordings, Clinical neurophysiology
118 (2007) 98–104.
[75] D. Moretti, F. Babiloni, F. Carducci, F. Cincotti, E. Remondini, P. Rossini, S. Salinari,
C. Babiloni, Computerized processing of eeg–eog–emg artifacts for multi-centric studies
in eeg oscillations and event-related potentials, International Journal of Psychophysiology
47 (2003) 199–216.
23
[76] S. Xing, R. McCardle, S. Xie, The development of eeg-based brain computer interfaces:
potential and challenges, International Journal of Computer Applications in Technology
50 (2014) 84–98.
[80] G. Huve, K. Takahashi, M. Hashimoto, Brain activity recognition with a wearable fnirs us-
ing neural networks, in: Mechatronics and Automation (ICMA), 2017 IEEE International
Conference on, IEEE, pp. 1573–1578.
[89] A. Bulling, J. A. Ward, H. Gellersen, G. Troster, Eye movement analysis for activity
recognition using electrooculography, IEEE transactions on pattern analysis and machine
intelligence 33 (2011) 741–753.
24
[90] R. Barea, L. Boquete, M. Mazo, E. López, System for assisted mobility using eye move-
ments based on electrooculography, IEEE transactions on neural systems and rehabilita-
tion engineering 10 (2002) 209–218.
[92] O. Faust, Documenting and predicting topic changes in computers in biology and medicine:
A bibliometric keyword analysis from 1990 to 2017, Informatics in Medicine Unlocked 11
(2018) 15 – 27.
[93] L. Fraiwan, K. Lweesy, Neonatal sleep state identification using deep learning autoen-
coders, in: Signal Processing & its Applications (CSPA), 2017 IEEE 13th International
Colloquium on, IEEE, pp. 228–231.
25
[103] U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, H. Adeli, Deep convolutional neural
network for the automated detection and diagnosis of seizure using eeg signals, Computers
in Biology and Medicine (2017).
[109] N. Lu, T. Li, X. Ren, H. Miao, A deep learning scheme for motor imagery classification
based on restricted boltzmann machines, IEEE Transactions on Neural Systems and
Rehabilitation Engineering 25 (2017) 566–576.
[111] P. Sajda, A. Gerson, K.-R. Muller, B. Blankertz, L. Parra, A data analysis competi-
tion to evaluate machine learning algorithms for use in brain-computer interfaces, IEEE
Transactions on neural systems and rehabilitation engineering 11 (2003) 184–185.
[113] M. Hajinoroozi, Z. Mao, T.-P. Jung, C.-T. Lin, Y. Huang, Eeg-based prediction of driver’s
cognitive performance by deep convolutional neural network, Signal Processing: Image
Communication 47 (2016) 549–555.
[115] L. Jingwei, C. Yin, Z. Weidong, Deep learning eeg response representation for brain
computer interface, in: Control Conference (CCC), 2015 34th Chinese, IEEE, pp. 3518–
3523.
26
[116] H. Piroska, S. Janos, Specific movement detection in eeg signal using time-frequency
analysis, in: Complexity and Intelligence of the Artificial and Natural Complex Systems,
Medical Applications of the Complex Systems, Biomedical Computing, 2008. CANS’08.
First International Conference on, IEEE, pp. 209–215.
[117] M. Hajinoroozi, Z. Mao, Y. Huang, Prediction of driver’s drowsy and alert states from
eeg signals with deep learning, in: Computational Advances in Multi-Sensor Adaptive
Processing (CAMSAP), 2015 IEEE 6th International Workshop on, IEEE, pp. 493–496.
[119] S. Koelstra, C. Muhl, M. Soleymani, J.-S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Ni-
jholt, I. Patras, Deap: A database for emotion analysis; using physiological signals, IEEE
Transactions on Affective Computing 3 (2012) 18–31.
[120] X. An, D. Kuang, X. Guo, Y. Zhao, L. He, A deep learning method for classification of
eeg data based on motor imagery, in: International Conference on Intelligent Computing,
Springer, pp. 203–210.
[121] W.-L. Zheng, J.-Y. Zhu, Y. Peng, B.-L. Lu, Eeg-based emotion classification using deep
belief networks, in: Multimedia and Expo (ICME), 2014 IEEE International Conference
on, IEEE, pp. 1–6.
[122] J. Li, A. Cichocki, Deep learning of multifractal attributes from motor imagery induced
eeg, in: International Conference on Neural Information Processing, Springer, pp. 503–510.
[123] X. Jia, K. Li, X. Li, A. Zhang, A novel semi-supervised deep learning framework for
affective state recognition on eeg signals, in: Bioinformatics and Bioengineering (BIBE),
2014 IEEE International Conference on, IEEE, pp. 30–37.
[124] Y. Ren, Y. Wu, Convolutional deep belief networks for feature extraction of eeg signal, in:
Neural Networks (IJCNN), 2014 International Joint Conference on, IEEE, pp. 2850–2853.
[127] H. Cecotti, A. Graeser, Convolutional neural network with embedded fourier transform
for eeg classification, in: Pattern Recognition, 2008. ICPR 2008. 19th International Con-
ference on, IEEE, pp. 1–4.
27
[130] J. H. Tan, Y. Hagiwara, W. Pang, I. Lim, S. L. Oh, M. Adam, R. San Tan, M. Chen, U. R.
Acharya, Application of stacked convolutional and long short-term memory network for
accurate identification of cad ecg signals, Computers in Biology and Medicine (2018).
[134] G. Zheng, S. Ji, M. Dai, Y. Sun, Ecg based identification by deep learning, in: Chinese
Conference on Biometric Recognition, Springer, pp. 503–510.
[135] G. B. Moody, R. G. Mark, The impact of the mit-bih arrhythmia database, IEEE
Engineering in Medicine and Biology Magazine 20 (2001) 45–50.
[136] A. Majumdar, R. Ward, Robust greedy deep dictionary learning for ecg arrhythmia
classification, in: Neural Networks (IJCNN), 2017 International Joint Conference on,
IEEE, pp. 4400–4407.
[138] K. Luo, J. Li, Z. Wang, A. Cuschieri, Patient-specific deep architectural model for ecg
classification, Journal of Healthcare Engineering 2017 (2017).
[140] S. D. Greenwald, The development and analysis of a ventricular fibrillation detector, Ph.D.
thesis, Massachusetts Institute of Technology, 1986.
[143] M. Cheng, W. J. Sori, F. Jiang, A. Khan, S. Liu, Recurrent neural network based classi-
fication of ecg signal features for obstruction of sleep apnea detection, in: Computational
Science and Engineering (CSE) and Embedded and Ubiquitous Computing (EUC), 2017
IEEE International Conference on, volume 2, IEEE, pp. 199–202.
28
[144] T. Penzel, G. B. Moody, R. G. Mark, A. L. Goldberger, J. H. Peter, The apnea-ecg
database, in: Computers in cardiology 2000, IEEE, pp. 255–258.
[148] Y. Zheng, Q. Liu, E. Chen, Y. Ge, J. L. Zhao, Time series classification using multi-
channels deep convolutional neural networks, in: International Conference on Web-Age
Information Management, Springer, pp. 298–310.
[149] L.-H. Du, W. Liu, W.-L. Zheng, B.-L. Lu, Detecting driving fatigue with multimodal deep
learning, in: Neural Engineering (NER), 2017 8th International IEEE/EMBS Conference
on, IEEE, pp. 74–77.
[150] J. Zhang, S. Li, R. Wang, Pattern recognition of momentary mental workload based
on multi-channel electrophysiological data and ensemble convolutional neural networks,
Frontiers in Neuroscience 11 (2017).
[153] X. Zhu, W.-L. Zheng, B.-L. Lu, X. Chen, S. Chen, C. Wang, Eog-based drowsiness
detection using convolutional neural networks., in: IJCNN, pp. 128–134.
[154] M. Längkvist, L. Karlsson, A. Loutfi, Sleep stage classification using unsupervised feature
learning, Advances in Artificial Neural Systems 2012 (2012) 5.
[157] S. Tokui, K. Oono, S. Hido, J. Clayton, Chainer: a next-generation open source framework
for deep learning, in: Proceedings of workshop on machine learning systems (LearningSys)
in the twenty-ninth annual conference on neural information processing systems (NIPS),
volume 5, pp. 1–6.
[158] K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman, Return of the devil in the details:
Delving deep into convolutional nets, arXiv preprint arXiv:1405.3531 (2014).
29
[159] D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, S. Bengio, Why does
unsupervised pre-training help deep learning?, Journal of Machine Learning Research 11
(2010) 625–660.
[160] X.-W. Chen, X. Lin, Big data deep learning: challenges and perspectives, IEEE access 2
(2014) 514–525.
[164] B. Zadrozny, Learning and evaluating classifiers under sample selection bias, in: Proceed-
ings of the twenty-first international conference on Machine learning, ACM, p. 114.
[166] J.-G. Lee, S. Jun, Y.-W. Cho, H. Lee, G. B. Kim, J. B. Seo, N. Kim, Deep learning in
medical imaging: general overview, Korean journal of radiology 18 (2017) 570–584.
.
Table A.8: Author supplied keywords mapped to topics
30
algorithm ada-boost, algorithm, algorithms, classification algorithms, component
analysis, compression, hidden markov models, independent component
analysis
dbn belief networks, boltzmann machines, convolutional deep belief net-
works, dbn, deep belief network, rbm, restricted boltzman machine
(rbm), restricted boltzmann machine, deep neural network, deep neural
networks
eeg brain signals, eeg, eeg data, eeg-based hand squeeze task, electroen-
cephalography, electroencephalography (eeg), encephalogram signals
feature feature clustering, feature extraction, feature learning, feature-
extraction, features, unsupervised feature learning
learning deep feature learning, deep learning, ensemble learning
machine machine, machines
model mathematical model, model, models
motor imagery motor imagery
sleep apnea detection, automatic sleep stage classification, obstructive sleep
apnea detection, sleep apnea, sleep stage classification, slow eye-
movements
classification classification, emotion classification, patient-specific ecg classification,
pattern classification
control multifunction myoelectric control, myoelectric control
emg electromyogram, electromyography, emg-based speech recognition, non-
stationary emg, surface emg
ml machine learning, machine learning algorithms
prostheses prostheses, prosthetics, upper-limb, upper-limb prostheses
recognition brain activity recognition, pattern recognition, pattern-recognition,
recognition, signal recognition
signal medical signal detection, medical signal processing, signal classification,
signal reconstruction, signal representation, signals, time preserving sig-
nal representation strategy
bci bci brain computer interface, brain computer interface (bci), brain
computer interfaces, brain-computer interface, brain-computer interface
(bci), brain-computer interfaces
cnn cnn, convolution, convolution neural network, convolutional neural net-
work, convolutional neural networks, convolutional neural networks
(cnns)
epilepsy epilepsy, seizure
nn batch normalization layers, biological neural networks, feedforward neu-
ral nets, neural-networks, neurons
real-time real-time, real-time heart monitoring, real-time systems, truenorth-
enabled real-time classification
system ibm truenorth neurosynaptic system, low-power platform, system
31