1. Introduction
Epilepsy is caused by the abnormal electrical activities of neurons in the brain, leading to convulsions and loss of consciousness. According to the report published by the World Health Organization (WHO), epilepsy has become one of the most common neurological diseases globally, affecting about 50 million people worldwide [
1]. Epileptic patients are three times more likely to die prematurely because of the accidental injuries or the brain damage caused by continuous seizures when compared with the general population. Therefore, having an effective framework to predict epileptic seizures is significant for epileptic patients.
In recent years, Internet of Things (IoT) technology has been flourishing and made remarkable achievements in smart home [
2,
3], intelligent logistics [
4] and intelligent transportation [
5]. At the same time, smart healthcare is deeply integrated with the IoT, artificial intelligence and the medical industry [
6,
7], providing a new framework for hospitals, doctors and patients. Telemedicine and the diagnosis of chronic diseases with wearable devices, such as epilepsy, diabetes and heart disease, are smart healthcare. Doctors can monitor patients’ health status in real time through IoT technology and wearable devices. Academia is also committed to researching and explore IoT solutions for seizure prediction [
8,
9]. The development of wireless technologies has ensured low latency [
10,
11] and power efficiency [
12,
13] for IoT networks. Artificial intelligence techniques can classify the EEG signals recorded in real time to achieve effective prediction of seizures.
In this study, a latest seizure prediction framework is suggested based on the IoT network and deep learning. We proposed a novel Synchrosqueezed Wavelet Transform (SWT) and Multi-Level Feature Convolutional Neural Network (MLF-CNN) system to predict seizure onset from Electroencephalogram (EEG) signals. EEG measures voltage fluctuations generated between neurons in the brain, which is the most common signal for monitoring brain state and widely used in the diagnosis of epilepsy. The SWT was used to measure the energy changes in EEG signals caused by seizures within a well-defined TF plane. The MLF-CNN model was used to extract multi-level features from the processed EEG signals and classify the different seizure segments. IoT technology enabled real-time monitoring, transmission, and recording of EEG signals from epileptic patients, and it provided effective seizure prediction together with deep learning as previously mentioned.
Currently, the prediction of epilepsy mainly focuses on measuring changes of the EEG signals before a seizure onset, known as the preictal. A lot of research has been carried out using the artificial intelligence approach for the prediction and classification of epileptic seizures. Therefore, many artificial intelligence algorithms have been developed, which can be divided into two categories: machine learning and deep learning. These algorithms are generally based on advanced signal processing techniques and feature extraction schemes.
Traditional machine learning methods require manual feature extraction, which significantly increases labor costs and human errors. Deep learning has excellent applications in various fields. For example, the AlphaGo from Deep Mind is based on a variety of deep neural network technology [
14]. Therefore, deep learning methods for predicting epileptic seizures are gaining more and more attention. One of the characteristics of deep learning is that relevant features can be extracted automatically using a network model. Zhou et al. [
15] used the Fast Fourier Transform (FFT) and Convolutional Neural Network (CNN) model to predict the onset of epilepsy. Their proposed model achieved an accuracy of 97.7% on the Freiburg dataset and 91.1% on the CHB-MIT dataset. They also concluded that the performance of the frequency domain is better than that of the time domain. Shahbazi et al. [
16] applied Short-Time Fourier Transform (STFT) to the EEG signals to construct a multichannel EEG image. A three-layer CNN was also used to extract spatial features from STFT images. Additionally, a Long-Short Time Memory network (LSTM) and a post-processing procedure were utilized to automatically extract temporal features from the EEG signal, reducing the impact of individual prediction errors. This algorithm achieved a sensitivity of 98.21% and a false prediction rate of 0.13 FPR/h on the CHB-MIT dataset. Truong et al. [
17] applied the STFT algorithm to extract time and frequency domain information from a 30-s EEG window. The extracted information was used to train a three-layer CNN network, which achieved a sensitivity of 81.4%, 81.2% and 75% on the Freiburg Hospital intracranial EEG dataset, CHB-MIT dataset and the American Epilepsy Society Seizure Prediction Challenge dataset, respectively. In addition, Liu et al. [
18] used both frequency and time domains to develop their prediction algorithm. The time-domain data were processed using principal component analysis (PCA). The frequency-domain data were obtained using FFT. The final algorithm was tested on two cases obtained from the CHB-MIT dataset, and an area under the curve (AUCs) of 0.85.
The wavelet transform method is another method commonly used for time–frequency analysis. Various methods have been proposed to transform the EEG to predict the onset of epileptic seizures. Khan et al. [
19] used a seven-layer CNN to predict the onset of epilepsy. This method achieved a sensitivity of 87.8% and a false prediction rate of 0.142 FP/h. Hussein [
20] used the Continuous Wavelet Transform (CWT) to transform the EEG signal into an image-like format. A Semi-Dilated Convolutional Network (SDCN) was then used to expand the receiver in time domain while maintaining the same resolution within the frequency domain. Aliyu et al. [
21] used Discrete Wavelet Transform (DWT) to remove noise and selected the most relevant prediction features from 20 extracted features. The features that were significantly correlated with the onset of epilepsy were identified and used to train a LSTM model. This method significantly reduced the parameters required to train the model and improved the model’s performance. Shoeibi et al. [
22] used tunable-Q wavelet transform (TQWT) to decompose the EEG signals into different sub-bands, and 15 different fuzzy entropies were extracted from 9 sub-bands of TQWT. By an autoencoder and breeding swarm optimization (ANFIS-BS) method, they obtained an fantastic accuracy of 99.74% in classifying into two classes on the Bonn dataset.
Other studies [
23,
24,
25,
26,
27] used the raw EEG signals without any preprocessing and also achieved good results in predicting epileptic seizures. Besides, Cao and Hu et al. [
28,
29,
30,
31] achieved multi level prediction of epilepsy and obtained good results using the Mean Amplitude Spectrum (MAS). Ozdemir [
32] proposed a novel method based on Fourier-based Synchrosqueezing Transform, to optimize a ResNet50 network, and also achieved good results in both epilepsy detection and prediction.
The two main factors affecting the prediction accuracy of current algorithms were effective signal processing methods and the development of an efficient deep learning model. Conventional Fourier transform and wavelet transform have a limited resolution as they are constrained by the uncertainty principle. According to this principle, the frequency resolution decreases when the temporal resolution increases and, conversely, the temporal resolution decreases when the frequency resolution increases. Therefore, to resolve these issues, SWT-based CNN prediction model is introduced and integrated into a smart IoT network framework for epileptic seizure. The contributions of this paper are as follows.
- (1)
A smart epileptic seizure prediction IoT framework using deep learning technology was proposed. It can provide services for hospitals, doctors and patients. For epileptic patients who cannot be operated on, it can send warnings before epileptic seizures, reduce psychological pressures and improve their quality of life.
- (2)
Synchrosqueezed Wavelet Transform (SWT) was introduced to process and analyze EEG signals. SWT is a signal rearrangement algorithm based on wavelet transform in the time–frequency domain. It can clearly represent the sudden energy discharges and provide highly localized time–frequency energy distributions of EEG signals, which can help to improve the classification accuracy.
- (3)
A novel Multi-Level Feature Convolutional Neural Network (MLF-CNN) was established to extract features from different dimensions automatically. The extracted features were concatenated and sent to a hierarchical neural network which consisted of 2 fully-connected layers, 2 dropout layers followed by a softmax layer developed for EEG feature learning and epileptic states classification. The proposed MLF-CNN achieved better performance compared with some other research, which was demonstrated in
Section 4.
- (4)
The proposed epileptic seizure prediction system was validated not only on the public CHB-MIT dataset but also on the private ZJU4H dataset collected by our cooperation hospital. Great performance was achieved by both of them. This demonstrated that our approach could provide robust epilepsy prediction.
The remainder of this paper is organized as follows.
Section 2 describes the entire experimental process, including the datasets used for the study, development of the SWT algorithm and data preprocessing.
Section 3 discusses the development of the MLF-CNN prediction model.
Section 4 describes the performance of our proposed model using two datasets, which we collected ZJU4H dataset at hospital. Finally, in
Section 5, the research findings are summarized and recommendations for further research are given.
4. Performance Analysis
The algorithm presented in this study was based on the time–frequency analysis method illustrated in
Section 2. All training and testing procedures were executed on an RTX TITAN graphics processing unit with 32G random access memory using Python 3.6, Keras 2.3.0, and TensorFlow 2.2.0.
Two EEG segment lengths and two models were used for training and testing. The different lengths were used to investigate the effect of the signal segment size on the experimental results. The performance of our model in predicting the onset of epilepsy was compared with CWT. The evaluation metrics used in this paper were accuracy, sensitivity, and specificity (also called recall). Sensitivity is the percentage of true positive samples in the actual positive samples, and specificity is the percentage of true negative samples in the actual negative samples. If the sensitivity of the proposed model is too low, the rate of false negatives will increase. False negatives may result in a delay in the provision of treatment. Conversely, if specificity is poor, the false positive rate will increase. False positives could lead to unnecessary treatment interventions and increase anxiety levels amongst patients. Therefore, the epilepsy algorithm needs to have a high sensitivity and specificity to be feasible for clinical use.
All the 3 s segments were trained using 5-fold cross-validation to reduce the risk of overfitting the model.
Figure 5. shows the accuracy and loss changes during the training of 3 s SWT segments. It is important to note that the accuracy values converge at the upper limit, and the loss values converge at the lower limit. Therefore, no overfitting or underfitting is observed in any of the evaluated models. Although the SWT-3s model converged before the 50 epochs, the epoch was still fixed at 50 to control the variables and maintain the consistency of the experimental conditions.
Table 3 shows the training results based on the VGG16 model with CWT and SWT time–frequency images, which were obtained from each patient. For a 1 s-duration segment, the highest prediction performance was achieved by both the CWT images and SWT images under the VGG16 model for patient chb08. However, large variations in the prediction performance of both the CWT and SWT images were noted. For example, in the CWT image for patient chb04, the accuracy and specificity were less than 60.00%, but the sensitivity reached 93.52%. Similarly, for the SWT image of patient chb10, the accuracy, sensitivity, and specificity were 68.20%, 72.24%, and 65.41%, respectively. The large variations in the algorithm’s prediction accuracy for epilepsy in different patients could increase the rate of false predictions leading to late or inappropriate interventions by healthcare professionals.
To compare the effect of different segment lengths on the experimental results, we repeated the above experiments using 3 s-duration segments, and the detailed results are shown in
Table 4. Since the 3 s segments contained more information than the 1 s segment, the overall average performances of both CWT and SWT improved. The average accuracy, sensitivity, and specificity for CWT were 93.84%, 93.47%, and 94.21%, respectively. Similarly, the average accuracy, sensitivity, and specificity for SWT were 94.99%, 94.50%, and 95.25%, respectively. The CWT algorithm achieved very high sensitivity and specificity for most patients, and eventually, four patients (chb01, chb11, chb19, chb20) achieved a 100% result. However, the model still performed poorly in some patients, such as in patient chb02 with a specificity of 63.92% and patient chb04 with a sensitivity of 50.00%. Similarly, although the overall performance of SWT was improved compared with CWT, the specificity of patient chb16 and the sensitivity of patient chb21 were still less than satisfactory.
To address the problem of reasonable accuracy but poor sensitivity and specificity in some patients, we proposed the MLF-CNN model, which is detailed in
Section 3. Since, overall, the performance of SWT was better than that of CWT, only the 1 s and 3 s SWT segments were integrated into the MLF-CNN model for further testing. The test results are shown in
Table 5. The performance of the MLF-CNN model improved compared with VGG16 and eventually achieved an overall accuracy, sensitivity, and specificity of 96.99%, 96.48%, and 97.46%. Furthermore, the model provided more stable sensitivity and specificity results and achieved a false prediction rate of 0.031 FPR/h. For the 1 s-duration SWT, the lowest sensitivity was 78.75% (chb06) and the lowest specificity was 74.17% (chb10). Moreover, the results of all metrics were improved with the 3 s-duration SWT. The lowest prediction accuracy for the 3 s-duration SWT was 85.00% (chb14). The difference in sensitivity and specificity was less than 3%.
The results of the above experiments using SWT were integrated and plotted as a graph shown in
Figure 6. The findings indicate that the 3 s segment provided more information and effectively improved the prediction performance when compared with the 1 s segment. The proposed MLF-CNN prediction algorithm further improved the accuracy, sensitivity, and specificity of the time–frequency classification of epilepsy on the EEG signal. Furthermore, the prediction results were more stable, as shown in
Figure 7.
In addition, we used the ZJU4H dataset to validate the MLF-CNN model. The experiment results are shown in
Table 6. However, only the 1 s segments were used for validation due to the limited data was available. The model’s overall accuracy, sensitivity, and specificity following validation were 94.25%, 97.76%, and 94.07%, respectively. It also achieved a false prediction rate of 0.049 FPR/h. This finding shows that our method can perform well on other datasets.
The literature findings evaluating the accuracy of classification-based seizure prediction models on the CHB-MIT dataset are summarized in
Table 7. Since the seizure prediction has already achieved high accuracy in recently published studies, we focused on sensitivity and specificity. A time–frequency map model using the discrete wavelet transform as input to a five-layer CNN [
19] achieved an average sensitivity of 87.8% and specificity of 85.8% on the CHB-MIT dataset. Truong proposed method achieved a sensitivity and specificity of less than 85% [
17]. Compared with previous works, our proposed method achieved the highest accuracy (96.99%), sensitivity (96.48%), and specificity (97.46%) on the CHB-MIT scalp dataset.
5. Conclusions and the Future Work
This paper proposed an intelligent epileptic prediction system based on Synchrosqueezed Wavelet Transform (SWT) and Multi-Level Feature Convolutional Neural Network (MLF-CNN) for Smart healthcare IoT network, in which the EEG data of epileptic patients are collected, transmitted and processed to provide prompt seizure onset alert for doctors and patients to take necessary measures. In the SWT based MLF-CNN model, time–frequency images of EEG segments were obtained using SWT, which provides a higher TF resolution of EEG signals. Then, the processed time–frequency images were used to train MLF-CNN model to extract multi-level feature information. The proposed system was tested on the CHB-MIT dataset and the ZJU4H dataset. To the best of our knowledge, this is the first study that used SWT to analyze epileptic EEG signals and to classify TF images using a CNN model. Our model achieved a high level of accuracy on both CHB-MIT and ZJU4H datasets. Furthermore, our model also achieved higher sensitivity and specificity when compared with other existing epilepsy prediction models.
Although the proposed method performed well, the sample size was relatively, small which probably limits the generalizability of the research findings. For the future work, novel signal processing techniques will be developed to characterize the EEG signals more effectively based on the premise of collecting more EEG data. Moreover, individual differences in the EEG signals among different patients are not well studied. More researches should evaluate these variations to further enhance the generalizability of the model. At last, it is worth noting that we adopted the CHB-MIT dataset and ZJU4H dataset, which mainly collected the data of young patients. Since most patients have their first onset in childhood, the prevention and treatment of children’s epilepsy is particularly important. Significantly, the status of adult and elderly patients should also be reviewed, and we will consider them in the future work.