Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

TS-GAN: Time-series GAN for Sensor-based Health Data Augmentation

Published: 18 April 2023 Publication History

Abstract

Deep learning has achieved significant success on intelligent medical treatments, such as automatic diagnosis and analysis of medical data. To train an automatic diagnosis system with high accuracy and strong robustness in healthcare, sufficient training data are required when using deep learning-based methods. However, given that the data collected by sensors that are embedded in medical or mobile devices are inadequate, it is challenging to train an effective and efficient classification model with state-of-the-art performance. Inspired by generative adversarial networks (GANs), we propose TS-GAN, a Time-series GAN architecture based on long short-term memory (LSTM) networks for sensor-based health data augmentation, thereby improving the performance of deep learning-based classification models. TS-GAN aims to learn a generative model that creates time-series data with the same space and time dependence as the real data. Specifically, we design an LSTM-based generator for creating realistic data and an LSTM-based discriminator for determining how similar the generated data are to real data. In particular, we design a sequential-squeeze-and-excitation module in the LSTM-based discriminator to better understand space dependence of real data, and apply the gradient penalty originated from Wasserstein GANs in the training process to stabilize the optimization. We conduct comparative experiments to evaluate the performance of TS-GAN with TimeGAN, C-RNN-GAN and Conditional Wasserstein GANs through discriminator loss, maximum mean discrepancy, visualization methods and classification accuracy on health datasets of ECG_200, NonInvasiveFatalECG_Thorax1, and mHealth, respectively. The experimental results show that TS-GAN exceeds other state-of-the-art time-series GANs in almost all the evaluation metrics, and the classifier trained on synthetic datasets generated by TS-GAN achieves the highest classification accuracy of 97.50% on ECG_200, 94.12% on NonInvasiveFatalECG_Thorax1, and 98.12% on mHealth, respectively.

References

[1]
Md Momin Al Aziz, Tanbir Ahmed, Tasnia Faequa, Xiaoqian Jiang, Yiyu Yao, and Noman Mohammed. 2021. Differentially private medical texts generation using generative neural networks. ACM Trans. Comput. Healthcare 3, 1, Article 5 (Oct.2021), 27 pages. DOI:
[2]
Talal Alshammari, Nasser Alshammari, Mohamed Sedky, and Chris Howard. 2018. SIMADL: Simulated activities of daily living dataset. Data 3, 2 (2018), 11.
[3]
Damla Arifoglu and Abdelhamid Bouchachia. 2019. Abnormal behaviour detection for dementia sufferers via transfer learning and recursive auto-encoders. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops’19). IEEE, 529–534.
[4]
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein generative adversarial networks. In Proceedings of the International Conference on Machine Learning. PMLR, 214–223.
[5]
Oresti Banos, Claudia Villalonga, Rafael Garcia, Alejandro Saez, Miguel Damas, Juan A. Holgado-Terriza, Sungyong Lee, Hector Pomares, and Ignacio Rojas. 2015. Design, implementation and validation of a novel open framework for agile development of mobile health applications. Biomed. Eng. Online 14, 2 (2015), 1–20. Retrieved from http://archive.ics.uci.edu/ml/datasets/mhealth+dataset.
[6]
Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexander Hammers, David Alexander Dickie, Maria Valdés Hernández, Joanna Wardlaw, and Daniel Rueckert. 2018. Gan augmentation: Augmenting training data using generative adversarial networks. Retrieved from https://arXiv:1810.10863.
[7]
Qili Chen, Bofan Liang, and Jiuhe Wang. 2019. A comparative study of LSTM and phased LSTM for gait prediction. Int. J. Artificial. Intelli. App. 10, 4 (2019), 57–66.
[8]
Tsz-Him Cheung and Dit-Yan Yeung. 2020. Modals: Modality-agnostic automated data augmentation in the latent space. In Proceedings of the International Conference on Learning Representations.
[9]
Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2018. Autoaugment: Learning augmentation policies from data. Retrieved from https://arXiv:1805.09501.
[10]
Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. 2020. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 702–703.
[11]
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. 2019. The UCR time-series archive. IEEE/CAA J. Autom. Sinica 6, 6 (2019), 1293–1305. Retrieved from https://www.cs.ucr.edu/eamonn/time_series_data_2018/.
[12]
Shaojiang Deng, Jiaxing Luo, and Yantao Li. 2021. CNN-based continuous authentication on smartphones with auto augmentation search. In Information and Communications Security, Debin Gao, Qi Li, Xiaohong Guan, and Xiaofeng Liao (Eds.). Springer International Publishing, Cham, 169–186.
[13]
Sander Dieleman, Kyle W. Willett, and Joni Dambre. 2015. Rotation-invariant convolutional neural networks for galaxy morphology prediction. Monthly Notices Roy. Astron. Soc. 450, 2 (2015), 1441–1459.
[14]
Zahra Ebrahimi, Mohammad Loni, Masoud Daneshtalab, and Arash Gharehbaghi. 2020. A review on deep learning methods for ECG arrhythmia classification. Expert Syst. Appl.: X 7 (2020), 100033.
[15]
Imanne El Maachi, Guillaume-Alexandre Bilodeau, and Wassim Bouachir. 2020. Deep 1D-convnet for accurate Parkinson disease detection and severity prediction from gait. Expert Syst. Appl. 143 (2020), 113075.
[16]
Farzan Farnia and Asuman Ozdaglar. 2020. Do GANs always have Nash equilibria? In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research), Hal Daume III and Aarti Singh (Eds.), Vol. 119. PMLR, 3029–3039.
[17]
Tonya Fields, George Hsieh, and Jules Chenou. 2019. Mitigating drift in time-series data with noise augmentation. In Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI’19). IEEE, 227–230.
[18]
Alexander L. Fogel and Joseph C. Kvedar. 2018. Artificial intelligence powers digital medicine. NPJ Dig. Med. 1, 1 (2018), 1–4.
[19]
Elizabeth Fons, Paula Dawson, Xiao-jun Zeng, John Keane, and Alexandros Iosifidis. 2021. Adaptive weighting scheme for automatic time-series data augmentation. Retrieved from https://arXiv:2102.08310.
[20]
Ian Goodfellow. 2016. Nips 2016 tutorial: Generative adversarial networks. Retrieved from https://arXiv:1701.00160.
[21]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.
[22]
Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18, 5-6 (2005), 602–610.
[23]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of wasserstein gans. Retrieved from https://arXiv:1704.00028.
[24]
Yang Guo, Dongsheng An, Xin Qi, Zhongxuan Luo, Shing-Tung Yau, Xianfeng Gu, et al. 2019. Mode collapse and regularity of optimal transportation maps. Retrieved from https://arXiv:1902.02934.
[25]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132–7141.
[26]
Amir Hussein, Marc Djandji, Reem A. Mahmoud, Mohamad Dhaybi, and Hazem Hajj. 2020. Augmenting DL with adversarial training for robust prediction of epilepsy seizures. ACM Trans. Comput. Healthcare 1, 3, Article 18 (June2020), 18 pages. DOI:
[27]
Navdeep Jaitly and Geoffrey E. Hinton. 2013. Vocal tract length perturbation (VTLP) improves speech recognition. In Proceedings of the International Conference on Machine Learning Workshop on Deep Learning for Audio, Speech and Language (ICML’13), Vol. 117. 21.
[28]
Madhura Joshi, Ankit Pal, and Malaikannan Sankarasubbu. 2022. Federated learning for healthcare domain - pipeline, applications and challenges. ACM Trans. Comput. Healthcare 3, 4 (2022), 36 pages.
[29]
Konrad Kluwak and Teodor Niżyński. 2020. Gait classification using LSTM networks for tagging system. In Proceedings of the IEEE 15th International Conference of System of Systems Engineering (SoSE’20). IEEE, 295–300.
[30]
Sayeri Lala, Maha Shady, Anastasiya Belyaeva, and Molei Liu. 2018. Evaluation of mode collapse in generative adversarial networks. In Proceedings of the IEEE High Performance Extreme Computing Conference (HPEC’18). IEEE, 1–9.
[31]
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. 2017. Mmd gan: Towards deeper understanding of moment matching network. Retrieved from https://arXiv:1705.08584.
[32]
Yantao Li, Hailong Hu, and Gang Zhou. 2018. Using data augmentation in continuous authentication on smartphones. IEEE Internet Things J. 6, 1 (2018), 628–640.
[33]
Yantao Li, Li Liu, Huafeng Qin, Shaojiang Deng, Mounim A. El-Yacoubi, and Gang Zhou. 2022. Adaptive deep feature fusion for continuous authentication with data augmentation. IEEE Trans. Mobile Comput. (2022), 1–16. DOI:
[34]
Yuan-Pin Lin, Yi-Hsuan Yang, and Tzyy-Ping Jung. 2014. Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening. Front Neurosci. 8 (2014), 14 pages.
[35]
Xinwen Liu, Huan Wang, Zongjin Li, and Lang Qin. 2021. Deep learning in ECG diagnosis: A review. Knowl.-Based Syst. 227 (2021), 107187.
[36]
Yifei Lu, Wei-Long Zheng, Binbin Li, and Bao-Liang Lu. 2015. Combining eye movements and EEG to enhance emotion recognition. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.
[37]
Yun Luo and Bao-Liang Lu. 2018. EEG data augmentation for emotion recognition using a conditional Wasserstein GAN. In Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’18). IEEE, 2535–2538.
[38]
Olof Mogren. 2016. C-RNN-GAN: Continuous recurrent neural networks with adversarial training. Retrieved from https://arXiv:1611.09904.
[39]
Daisuke Oba, Shinnosuke Matsuo, and Brian Kenji Iwana. 2021. Dynamic data augmentation with gating networks. Retrieved from https://arXiv:2111.03253.
[40]
Esteban Piacentino, Alvaro Guarner, and Cecilio Angulo. 2021. Generating synthetic ECGs using GANs for anonymizing healthcare data. Electronics 10, 4 (2021), 389.
[41]
Justin Salamon and Juan Pablo Bello. 2017. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Process. Lett. 24, 3 (2017), 279–283. DOI:
[42]
Arash Shaban-Nejad, Martin Michalowski, and David L. Buckeridge. 2018. Health intelligence: How artificial intelligence transforms population and personalized health. npj Digital Med 1 (2018), 2 pages.
[43]
Tamar Rott Shaham, Tali Dekel, and Tomer Michaeli. 2019. Singan: Learning a generative model from a single natural image. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4570–4580.
[44]
Terry T. Um, Franz M. J. Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. 2017. Data augmentation of wearable sensor data for Parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction. 216–220.
[45]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 86 (2008), 2579–2605.
[46]
Yong Xia, Naren Wulan, Kuanquan Wang, and Henggui Zhang. 2017. Atrial fibrillation detection using stationary wavelet transform and deep learning. In Proceedings of the Computing in Cardiology (CinC’17). IEEE, 1–4.
[47]
Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. 2019. Time-series generative adversarial networks. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS’19), Vol. 32, 1–11.
[48]
Chenshuang Zhang, Guijin Wang, Jingwei Zhao, Pengfei Gao, Jianping Lin, and Huazhong Yang. 2017. Patient-specific ECG classification based on recurrent neural networks and clustering technique. In Proceedings of the 13th IASTED International Conference on Biomedical Engineering (BioMed’17). IEEE, 63–67.
[49]
Zhen Zhao, Ze Li, Fuxin Li, and Yang Liu. 2021. CNN-LSTM based traffic prediction using spatial-temporal features. In Journal of Physics: Conference Series, Vol. 2037. IOP Publishing, 012065.
[50]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2223–2232.

Cited By

View all
  • (2024)Sleep Apnea Detection Using Wireless Wearable Single-Lead ECG: A One Dimensional Convolutional Neural Network ApproachDigital Health Research10.61499/dhr.2024.2.e32:2Online publication date: 2024
  • (2024)Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunitiesAIMS Public Health10.3934/publichealth.202400411:1(58-109)Online publication date: 2024
  • (2024)A Hybrid Deep Learning Approach with Generative Adversarial Network for Credit Card Fraud DetectionTechnologies10.3390/technologies1210018612:10(186)Online publication date: 2-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing for Healthcare
ACM Transactions on Computing for Healthcare  Volume 4, Issue 2
April 2023
154 pages
EISSN:2637-8051
DOI:10.1145/3592785
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2023
Online AM: 08 February 2023
Accepted: 26 January 2023
Revised: 19 December 2022
Received: 07 July 2022
Published in HEALTH Volume 4, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. GANs
  2. LSTM
  3. data augmentation
  4. SSE
  5. time dependence
  6. space dependence

Qualifiers

  • Research-article

Funding Sources

  • National Natural Science Foundation of China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,889
  • Downloads (Last 6 weeks)93
Reflects downloads up to 04 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Sleep Apnea Detection Using Wireless Wearable Single-Lead ECG: A One Dimensional Convolutional Neural Network ApproachDigital Health Research10.61499/dhr.2024.2.e32:2Online publication date: 2024
  • (2024)Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunitiesAIMS Public Health10.3934/publichealth.202400411:1(58-109)Online publication date: 2024
  • (2024)A Hybrid Deep Learning Approach with Generative Adversarial Network for Credit Card Fraud DetectionTechnologies10.3390/technologies1210018612:10(186)Online publication date: 2-Oct-2024
  • (2024)Opinion on enhancing diversity in photovoltaic scenario generation using weather data simulating by style-based generative adversarial networksFrontiers in Energy Research10.3389/fenrg.2024.140284012Online publication date: 21-May-2024
  • (2024)Lithium-ion battery application time series data augmentation based on generative adversarial network for training deep learning algorithm2024 IEEE 10th International Power Electronics and Motion Control Conference (IPEMC2024-ECCE Asia)10.1109/IPEMC-ECCEAsia60879.2024.10567686(4403-4408)Online publication date: 17-May-2024
  • (2024)From shallows to depths: unveiling hybrid synthetic data modeling for enhanced learning with privacy considerations in naturally imbalanced datasetsInternational Journal of Computers and Applications10.1080/1206212X.2024.2409989(1-16)Online publication date: 3-Oct-2024
  • (2024)Cross-sensor domain adaptation for data-driven correction of pedestrian level wind measurement in urban environments using cup anemometersSustainable Cities and Society10.1016/j.scs.2024.105430106(105430)Online publication date: Jul-2024
  • (2024)Augmenting time series data: An interpretable approach with metric learning and variational autoencodersInternational Journal of Electrical Power & Energy Systems10.1016/j.ijepes.2024.110190161(110190)Online publication date: Oct-2024
  • (2024)A systematic review of deep learning data augmentation in medical imaging: Recent advances and future research directionsHealthcare Analytics10.1016/j.health.2024.1003405(100340)Online publication date: Jun-2024
  • (2024)Synthetic data generation methods in healthcare: A review on open-source tools and methodsComputational and Structural Biotechnology Journal10.1016/j.csbj.2024.07.00523(2892-2910)Online publication date: Dec-2024
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media