research-article

Attentional Gated Res2Net for Multivariate Time Series Classification

Authors:

Guandong XuAuthors Info & Claims

Neural Processing Letters, Volume 55, Issue 2

Pages 1371 - 1395

https://doi.org/10.1007/s11063-022-10944-0

Published: 29 June 2022 Publication History

Abstract

Multivariate time series classification is a critical problem in data mining with broad applications. It requires harnessing the inter-relationship of multiple variables and various ranges of temporal dependencies to assign the correct classification label of the time series. Multivariate time series may come from a wide range of sources and be used in various scenarios, bringing the classifier challenge of temporal representation learning. We propose a novel convolutional neural network architecture called Attentional Gated Res2Net for multivariate time series classification. Our model uses hierarchical residual-like connections to achieve multi-scale receptive fields and capture multi-granular temporal information. The gating mechanism enables the model to consider the relations between the feature maps extracted by receptive fields of multiple sizes for information fusion. Further, we propose two types of attention modules, channel-wise attention and block-wise attention, to better leverage the multi-granular temporal patterns. Our experimental results on 14 benchmark multivariate time-series datasets show that our model outperforms several baselines and state-of-the-art methods by a large margin. Our model outperforms the SOTA by a large margin, the classification accuracy of our model is 10.16% better than the SOTA model. Besides, we demonstrate that our model improves the performance of existing models when used as a plugin. Further, based on our experiments and analysis, we provide practical advice on applying our model to a new problem.

References

[1]

Spiegel S, Gaebler J, Lommatzsch A, De Luca E, Albayrak S (2011) Pattern recognition and classification for multivariate time series. In: Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, pp. 34–42

[2]

Esling P and Agon C Time-series data mining ACM Computing Surveys (CSUR) 2012 45 1 1-34

[3]

Yu Z and Lee M Real-time human action classification using a dynamic neural model Neural Netw 2015 69 29-43

[4]

Chitra R and Seenivasagam V Heart disease prediction system using supervised learning classifier Bonfring Inter J Software Engineering Soft Comput 2013 3 1 01-07

[5]

Bai L, Yao L, Kanhere SS, Wang X, Yang Z (2018) Automatic device classification from network traffic streams of internet of things. In: 2018 IEEE 43rd Conference on Local Computer Networks (LCN), pp. 1–9. IEEE

[6]

Bengio Y, Courville A, and Vincent P Representation learning: A review and new perspectives IEEE Trans Pattern Anal Mach Intell 2013 35 8 1798-1828

[7]

Aydın S Deep learning classification of neuro-emotional phase domain complexity levels induced by affective video film clips IEEE J Biomed Health Inform 2019 24 6 1695-1702

[8]

Kılıç B, Aydın S (2022) Classification of contrasting discrete emotional states indicated by eeg based graph theoretical network measures. Neuroinformatics, 1–15

[9]

Aydın S (2021) Cross-validated adaboost classification of emotion regulation strategies identified by spectral coherence in resting-state. Neuroinformatics, 1–13

[10]

Baydogan MG, Runger G, and Tuv E A bag-of-features framework to classify time series IEEE Trans Pattern Anal Mach Intell 2013 35 11 2796-2802

[11]

Kampouraki A, Manis G, and Nikou C Heartbeat time series classification with support vector machines IEEE Trans Inf Technol Biomed 2008 13 4 512-518

[12]

Bai L, Yao L, Wang X, Kanhere SS, Xiao Y (2020) Prototype similarity learning for activity recognition. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 649–661. Springer

[13]

Bengio Y, LeCun Y, et al. Scaling learning algorithms towards ai Large-scale kernel machines 2007 34 5 1-41

[14]

LeCun Y, Bengio Y, and Hinton G Deep learning. nature 2015 521 7553 436-444

[15]

Hornik K Approximation capabilities of multilayer feedforward networks Neural Netw 1991 4 2 251-257

[16]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, and Polosukhin I Attention is all you need Advances in Neural Information Processing Systems 2017 30 5998-6008

[17]

Hochreiter S and Schmidhuber J Long short-term memory Neural Comput 1997 9 8 1735-1780

[18]

Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078

[19]

Xu P, Kumar D, Yang W, Zi W, Tang K, Huang C, Cheung JCK, Prince SJ, Cao Y (2021) Optimizing deeper transformers on small datasets. In: ACL/IJCNLP (1)

[20]

Di Gangi MA, Negri M, Cattoni R, Dessi R, Turchi M (2019) Enhancing transformer for end-to-end speech-to-text translation. In: Proceedings of Machine Translation Summit XVII: Research Track, pp. 21–31

[21]

Deng H, Runger G, Tuv E, and Vladimir M A time series forest for classification and feature extraction Inf Sci 2013 239 142-153

[22]

Jović A, Brkić K, Bogunović N (2012) Decision tree ensembles in biomedical time-series classification. In: Joint DAGM (German Association for Pattern Recognition) and OAGM Symposium, pp. 408–417. Springer

[23]

Zhang D, Zuo W, Zhang D, Zhang H (2010) Time series classification using support vector machine with gaussian elastic metric kernel. In: 2010 20th International Conference on Pattern Recognition, pp. 29–32. IEEE

[24]

Lee Y-H, Wei C-P, Cheng T-H, and Yang C-T Nearest-neighbor-based approach to time-series classification Decis Support Syst 2012 53 1 207-217

[25]

Bagnall A, Lines J, Bostrom A, Large J, and Keogh E The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances Data Min Knowl Disc 2017 31 3 606-660

[26]

Seto S, Zhang W, Zhou Y (2015) Multivariate time series classification using dynamic time warping template selection for human activity recognition. In: 2015 IEEE Symposium Series on Computational Intelligence, pp. 1399–1406. IEEE

[27]

Tang Y, Xu J, Matsumoto K, Ono C (2016) Sequence-to-sequence model with attention for time series classification. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 503–510. IEEE

[28]

Tan HX, Aung NN, Tian J, Chua MCH, and Yang YO Time series classification using a modified lstm approach from accelerometer-based data: A comparative study for gait cycle detection Gait & posture 2019 74 128-134

[29]

Elsayed N, Maida AS, Bayoumi M (2018) Deep gated recurrent and convolutional network hybrid model for univariate time series classification. arXiv preprint arXiv:1812.07683

[30]

Zhao B, Lu H, Chen S, Liu J, and Wu D Convolutional neural networks for time series classification J Syst Eng Electron 2017 28 1 162-169

[31]

Yang C, Jiang M, Guo Z, Liu Y (2020) Gated res2net for multivariate time series analysis. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE

[32]

Tang W, Long G, Liu L, Zhou T, Jiang J, Blumenstein M (2020) Rethinking 1d-cnn for time series classification: A stronger baseline. arXiv preprint arXiv:2002.10061

[33]

Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

[34]

Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448

[35]

Sun X, Wu P, and Hoi SC Face detection using deep learning: An improved faster rcnn approach Neurocomputing 2018 299 42-50

[36]

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer

[37]

He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969

[38]

Han Z, Zhao J, Leung H, Ma KF, Wang W (2019) A review of deep learning models for time series prediction. IEEE Sensors Journal

[39]

Borovykh A, Bohte S, Oosterlee CW (2017) Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691

[40]

Hoermann S, Bach M, Dietmayer K (2018) Dynamic occupancy grid prediction for urban autonomous driving: A deep learning approach with fully automatic labeling. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 2056–2063. IEEE

[41]

Ding X, Zhang Y, Liu T, Duan J (2015) Deep learning for event-driven stock prediction. In: Twenty-fourth International Joint Conference on Artificial Intelligence

[42]

Wallach I, Dzamba M, Heifets A (2015) Atomnet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv preprint arXiv:1510.02855

[43]

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9

[44]

Liu C-L, Hsaio W-H, and Tu Y-C Time series classification with multivariate convolutional neural network IEEE Trans Industr Electron 2018 66 6 4788-4797

[45]

Cui Z, Chen W, Chen Y (2016) Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995

[46]

Yang C, Jiang W, and Guo Z Time series data classification based on dual path cnn-rnn cascade network IEEE Access 2019 7 155304-155312

[47]

Karim F, Majumdar S, Darabi H, and Harford S Multivariate lstm-fcns for time series classification Neural Netw 2019 116 237-245

[48]

Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of AAAI

[49]

Rußwurm M and Körner M Self-attention for raw optical satellite time series classification ISPRS J Photogramm Remote Sens 2020 169 421-435

[50]

Hu J and Zheng W A deep learning model to effectively capture mutation information in multivariate time series prediction Knowl-Based Syst 2020 203

[51]

Woo S, Park J, Lee J-Y, So Kweon I (2018) Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19

[52]

Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141

[53]

Gao S, Cheng M-M, Zhao K, Zhang X-Y, Yang M-H, and Torr PH Res2net: A new multi-scale backbone architecture IEEE transactions on pattern analysis and machine intelligence 2019 43 2 652-662

[54]

Krizhevsky A, Sutskever I, and Hinton GE Imagenet classification with deep convolutional neural networks Adv Neural Inf Process Syst 2012 25 1097-1105

[55]

Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258

[56]

Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

[57]

Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3d points. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 9–14. IEEE

[58]

Schäfer P, Leser U (2017) Multivariate time series classification with weasel+ muse. arXiv preprint arXiv:1711.11343

[59]

Dau HA, Keogh E, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S (2018) Ratanamahatana: The UCR Time Series Classification Archive

[60]

Fawaz HI, Lucas B, Forestier G, Pelletier C, Schmidt DF, Weber J, Webb GI, Idoumghar L, Muller P-A, and Petitjean F Inceptiontime: Finding alexnet for time series classification Data Min Knowl Disc 2020 34 6 1936-1962

[61]

Middlehurst M, Large J, Bagnall A (2020) The canonical interval forest (cif) classifier for time series classification. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 188–195. IEEE

[62]

Müller M Dynamic time warping 2007 Berlin Springer 69-84

[63]

Dempster A, Petitjean F, and Webb GI Rocket: exceptionally fast and accurate time series classification using random convolutional kernels Data Min Knowl Disc 2020 34 5 1454-1495

[64]

Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

[65]

Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

[66]

Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z, Sun Y, He T, Mueller J, Manmatha R, Li M, Smola A (2020) ResNeSt: Split-Attention Networks

Cited By

Hong XHu JXu TRen XWu FMa XLi W(2024)MagNet: Multilevel Dynamic Wavelet Graph Neural Network for Multivariate Time Series ClassificationACM Transactions on Knowledge Discovery from Data10.1145/370391519:1(1-22)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3703915
Yang WXia KWang ZFan SLi L(2023)Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its ApplicationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.106151122:COnline publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.engappai.2023.106151

Recommendations

Stacking for multivariate time series classification

This work presents a novel approach to multivariate time series classification. The method exploits the multivariate structure of the time series and the possibilities of the stacking ensemble method. The basics of the method may be described in three ...
Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its Application
Abstract
Time Series Classification (TSC) in data mining is gradually developing as an important research direction. Many researchers have developed an extensive interest in Multivariate Time Series Classification (MTSC). The Self-Attention Causal Dilated ...
Multivariate LSTM-FCNs for time series classification
Abstract
Over the past decade, multivariate time series classification has received great attention. We propose transforming the existing univariate time series classification models, the Long Short Term Memory Fully Convolutional Network (LSTM-...

Comments

Information & Contributors

Information

Published In

cover image Neural Processing Letters

Neural Processing Letters Volume 55, Issue 2

Apr 2023

1087 pages

ISSN:1370-4621

Issue’s Table of Contents

© The Author(s) 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 29 June 2022

Accepted: 20 June 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council
Australian Research Council
University of Technology Sydney

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hong XHu JXu TRen XWu FMa XLi W(2024)MagNet: Multilevel Dynamic Wavelet Graph Neural Network for Multivariate Time Series ClassificationACM Transactions on Knowledge Discovery from Data10.1145/370391519:1(1-22)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3703915
Yang WXia KWang ZFan SLi L(2023)Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its ApplicationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.106151122:COnline publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.engappai.2023.106151

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents