Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3534678.3539117acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Learning Sparse Latent Graph Representations for Anomaly Detection in Multivariate Time Series

Published: 14 August 2022 Publication History

Abstract

Anomaly detection in high-dimensional time series is typically tackled using either reconstruction- or forecasting-based algorithms due to their abilities to learn compressed data representations and model temporal dependencies, respectively. However, most existing methods disregard the relationships between features, information that would be extremely useful when incorporated into a model. In this work, we introduce Fused Sparse Autoencoder and Graph Net (FuSAGNet), which jointly optimizes reconstruction and forecasting while explicitly modeling the relationships within multivariate time series. Our approach combines Sparse Autoencoder and Graph Neural Network, the latter of which predicts future time series behavior from sparse latent representations learned by the former as well as graph structures learned through recurrent feature embedding. Experimenting on three real-world cyber-physical system datasets, we empirically demonstrate that the proposed method enhances the overall anomaly detection performance, outperforming baseline approaches. Moreover, we show that mining sparse latent patterns from high-dimensional time series improves the robustness of the graph-based forecasting model. Lastly, we conduct visual analyses to investigate the interpretability of both recurrent feature embeddings and sparse latent representations.

Supplemental Material

MP4 File
This video contains a high-level description of the paper "Learning Sparse Latent Graph Representations for Anomaly Detection in Multivariate Time Series", which has been accepted for publication at the Applied Data Science Track of KDD 2022. The authors propose Fused Sparse Autoencoder and Graph Net (FuSAGNet), which jointly optimizes reconstruction by Sparse Autoencoder and forecasting by Graph Neural Network, to detect anomalies in noisy, high-dimensional time series generated from sensors in water treatment plants. Note that the content in the presentation video does not touch upon all technical details of the proposed method and the authors kindly refer anyone interested in further details to our full paper.

References

[1]
Charu C Aggarwal. 2015. Outlier analysis. In Data mining. Springer, 237--263.
[2]
Chuadhry Mujeeb Ahmed, Venkata Reddy Palleti, and Aditya P Mathur. 2017. WADI: a water distribution testbed for research in the design of secure cyber physical systems. In Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks. 25--28.
[3]
Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE 2, 1 (2015), 1--18.
[4]
Fabrizio Angiulli and Clara Pizzuti. 2002. Fast outlier detection in high dimensional spaces. In European conference on principles of data mining and knowledge discovery. Springer, 15--27.
[5]
Julien Audibert, Pietro Michiardi, Frédéric Guyard, Sébastien Marti, and Maria A Zuluaga. 2020. USAD: UnSupervised Anomaly Detection on Multivariate Time Series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3395--3404.
[6]
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Velickovic. 2021. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 (2021).
[7]
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine 34, 4 (2017), 18--42.
[8]
Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR) 41, 3 (2009), 1--58.
[9]
Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014).
[10]
Ailin Deng and Bryan Hooi. 2021. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada. 2--9.
[11]
Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).
[12]
Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. 2016. A dataset to support research in the design of secure water treatment systems. In International conference on critical information infrastructures security. Springer, 88--99.
[13]
Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, DavidWarde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014).
[14]
Sepp Hochreiter. 1998. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6, 02 (1998), 107--116.
[15]
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. 2018. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 387--395.
[16]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[17]
Thomas N Kipf and MaxWelling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
[18]
Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79--86.
[19]
Honglak Lee, Chaitanya Ekanadham, and Andrew Ng. 2007. Sparse deep belief net model for visual area V2. Advances in neural information processing systems 20 (2007), 873--880.
[20]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. 2019. MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In International Conference on Artificial Neural Networks. Springer, 703--716.
[21]
Feng Li, JM Zuraday, andWeiWu. 2018. SPARSE REPRESENTATION LEARNING OF DATA BY AUTOENCODERS WITH ??1/2 REGULARIZATION. Neural Network World 28, 2 (2018), 133--147.
[22]
Andrew Ng et al. 2011. Sparse autoencoder. CS294A Lecture notes 72, 2011 (2011), 1--19.
[23]
Keiron O'Shea and Ryan Nash. 2015. An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458 (2015).
[24]
Daehyung Park, Yuuna Hoshi, and Charles C Kemp. 2018. A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation Letters 3, 3 (2018), 1544--1551.
[25]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).
[26]
Marco AF Pimentel, David A Clifton, Lei Clifton, and Lionel Tarassenko. 2014. A review of novelty detection. Signal Processing 99 (2014), 215--249.
[27]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1985. Learning internal representations by error propagation. Technical Report. California Univ San Diego La Jolla Inst for Cognitive Science.
[28]
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. 2001. Estimating the support of a high-dimensional distribution. Neural computation 13, 7 (2001), 1443--1471.
[29]
Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing 45, 11 (1997), 2673--2681.
[30]
Hyeok-Ki Shin, Woomyo Lee, Jeong-Han Yun, and HyoungChun Kim. 2020. {HAI} 1.0: HIL-based Augmented {ICS} Security Dataset. In 13th {USENIX} Workshop on Cyber Security Experimentation and Test ({CSET} 20).
[31]
Mei-Ling Shyu, Shu-Ching Chen, Kanoksri Sarinnapakorn, and LiWu Chang. 2003. A novel anomaly detection scheme based on principal component classifier. Technical Report. MIAMI UNIV CORAL GABLES FL DEPT OF ELECTRICAL AND COMPUTER ENGINEERING.
[32]
Lu-An Tang, Jiawei Han, and Guofei Jiang. 2014. Mining sensor data in cyberphysical systems. Tsinghua Science and Technology 19, 3 (2014), 225--234.
[33]
Xianfeng Tang, Yandong Li, Yiwei Sun, Huaxiu Yao, Prasenjit Mitra, and Suhang Wang. 2020. Transferring robustness for graph neural network against poisoning attacks. In Proceedings of the 13th International Conference on Web Search and Data Mining. 600--608.
[34]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
[35]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).
[36]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[37]
John Wright, Allen Y Yang, Arvind Ganesh, S Shankar Sastry, and Yi Ma. 2008. Robust face recognition via sparse representation. IEEE transactions on pattern analysis and machine intelligence 31, 2 (2008), 210--227.
[38]
Hang Zhao, Yujing Wang, Juanyong Duan, Congrui Huang, Defu Cao, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, and Qi Zhang. 2020. Multivariate time-series anomaly detection via graph attention network. arXiv preprint arXiv:2009.02040 (2020).
[39]
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI Open 1 (2020), 57--81.
[40]
Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International Conference on Learning Representations.

Cited By

View all
  • (2024)Time-Series Anomaly Detection Based on Dynamic Temporal Graph Convolutional Network for Epilepsy DiagnosisBioengineering10.3390/bioengineering1101005311:1(53)Online publication date: 5-Jan-2024
  • (2024)Deep Learning for Time Series Anomaly Detection: A SurveyACM Computing Surveys10.1145/3691338Online publication date: 30-Aug-2024
  • (2024)An Explore–Exploit Workload-Bounded Strategy for Rare Event Detection in Massive Energy Sensor Time SeriesACM Transactions on Intelligent Systems and Technology10.1145/365764115:4(1-25)Online publication date: 28-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2022
5033 pages
ISBN:9781450393850
DOI:10.1145/3534678
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. anomaly detection
  2. graph neural network
  3. joint optimization
  4. multivariate time series
  5. sparse representations

Qualifiers

  • Research-article

Funding Sources

  • IITP by Korean MSIT (Artificial Intelligence Innovation Hub)
  • Basic Science Research Program through NRF by Korean MSIT
  • IITP by Korean MSIT (Graduate School of Convergence Security at Sungkyunkwan University)
  • IITP by Korean MSIT (Self-directed Multi-Modal Intelligence for solving unknown, open domain problems)
  • IITP by Korean MSIT (AI Platform to Fully Adapt and Reflect Privacy-Policy Changes)
  • IITP by Korean MSIT (AI Graduate School Support Program at Sungkyunkwan University)

Conference

KDD '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)998
  • Downloads (Last 6 weeks)39
Reflects downloads up to 30 Aug 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Time-Series Anomaly Detection Based on Dynamic Temporal Graph Convolutional Network for Epilepsy DiagnosisBioengineering10.3390/bioengineering1101005311:1(53)Online publication date: 5-Jan-2024
  • (2024)Deep Learning for Time Series Anomaly Detection: A SurveyACM Computing Surveys10.1145/3691338Online publication date: 30-Aug-2024
  • (2024)An Explore–Exploit Workload-Bounded Strategy for Rare Event Detection in Massive Energy Sensor Time SeriesACM Transactions on Intelligent Systems and Technology10.1145/365764115:4(1-25)Online publication date: 28-Jun-2024
  • (2024)Graph Time-series Modeling in Deep Learning: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/363853418:5(1-35)Online publication date: 28-Feb-2024
  • (2024)Correlation-Aware Spatial–Temporal Graph Learning for Multivariate Time-Series Anomaly DetectionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.332566735:9(11802-11816)Online publication date: Sep-2024
  • (2024)ICS Anomaly Detection Based on Sensor Patterns and Actuator Rules in Spatiotemporal DependencyIEEE Transactions on Industrial Informatics10.1109/TII.2024.339352820:8(10647-10656)Online publication date: Aug-2024
  • (2024)Coupled Attention Networks for Multivariate Time Series Anomaly DetectionIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2023.328057712:1(240-253)Online publication date: Jan-2024
  • (2024)Finding Component Relationships: A Deep-Learning-Based Anomaly Detection InterpreterIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.336043511:3(4149-4162)Online publication date: Jun-2024
  • (2024)Anomaly Detection via Graph Attention Networks-Augmented Mask Autoregressive Flow for Multivariate Time SeriesIEEE Internet of Things Journal10.1109/JIOT.2024.336239811:11(19368-19379)Online publication date: 1-Jun-2024
  • (2024)Global–Local Association Discrepancy for Multivariate Time Series Anomaly Detection in IIoTIEEE Internet of Things Journal10.1109/JIOT.2023.333069611:7(11287-11297)Online publication date: 1-Apr-2024
  • Show More Cited By

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media