Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Graph spatiotemporal process for multivariate time series anomaly detection with missing values

Published: 25 June 2024 Publication History

Abstract

The detection of anomalies in multivariate time series data is crucial for various practical applications, including smart power grids, traffic flow forecasting, and industrial process control. However, real-world time series data is usually not well-structured, posting significant challenges to existing approaches: (1) The existence of missing values in multivariate time series data along variable and time dimensions hinders the effective modeling of interwoven spatial and temporal dependencies, resulting in important patterns being overlooked during model training; (2) Anomaly scoring with irregularly-sampled observations is less explored, making it difficult to use existing detectors for multivariate series without fully-observed values. In this work, we introduce a novel framework called GST-Pro, which utilizes a graph spatiotemporal process and anomaly scorer to tackle the aforementioned challenges in detecting anomalies on irregularly-sampled multivariate time series. Our approach comprises two main components. First, we propose a graph spatiotemporal process based on neural controlled differential equations. This process enables effective modeling of multivariate time series from both spatial and temporal perspectives, even when the data contains missing values. Second, we present a novel distribution-based anomaly scoring mechanism that alleviates the reliance on complete uniform observations. By analyzing the predictions of the graph spatiotemporal process, our approach allows anomalies to be easily detected. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods, regardless of whether there are missing values present in the data. Our code is available: https://github.com/huankoh/GST-Pro.

Highlights

This paper presents a novel anomaly detection method for multivariate time series with missing values.
The method employs graph neural networks to capture both spatial and temporal dependencies.
A distribution-based anomaly scoring mechanism is used to detect anomalies.

References

[1]
Lee C., Luo Z., Ngiam K.Y., Zhang M., Zheng K., Chen G., Ooi B.C., Yip W.L.J., Big healthcare data analytics: Challenges and applications, in: Handbook of Large-Scale Distributed Computing in Smart Healthcare, Springer, 2017, pp. 11–41.
[2]
Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, D. Pei, Robust anomaly detection for multivariate time series through stochastic recurrent neural network, in: KDD, 2019, pp. 2828–2837.
[3]
Z. Li, Y. Zhao, J. Han, Y. Su, R. Jiao, X. Wen, D. Pei, Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding, in: KDD, 2021, pp. 3220–3230.
[4]
Mathur A.P., Tippenhauer N.O., SWaT: A water treatment testbed for research and training on ICS security, in: 2016 International Workshop on Cyber-Physical Systems for Smart Water Networks, IEEE, 2016, pp. 31–36.
[5]
K. Hundman, V. Constantinou, C. Laporte, I. Colwell, T. Soderstrom, Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding, in: KDD, 2018, pp. 387–395.
[6]
Garg A., Zhang W., Samaran J., Savitha R., Foo C.-S., An evaluation of anomaly detection and diagnosis in multivariate time series, IEEE TNNLS (2021).
[7]
Schmidl S., Wenig P., Papenbrock T., Anomaly detection in time series: a comprehensive evaluation, Proc. VLDB Endow. 15 (9) (2022) 1779–1797.
[8]
Darban Z.Z., Webb G.I., Pan S., Aggarwal C.C., Salehi M., Deep learning for time series anomaly detection: A survey, 2022, arXiv preprint arXiv:2211.05244.
[9]
Yu Q., Jibin L., Jiang L., An improved ARIMA-based traffic anomaly detection algorithm for wireless sensor networks, Int. J. Distrib. Sens. Netw. 12 (1) (2016).
[10]
Keogh E., Lin J., Fu A., Hot sax: Efficiently finding the most unusual time series subsequence, in: ICDM, Ieee, 2005, pp. 8–pp.
[11]
Ting K.M., Xu B.-C., Washio T., Zhou Z.-H., Isolation distributional kernel a new tool for point & group anomaly detection, IEEE TKDE (2021).
[12]
Park D., Hoshi Y., Kemp C.C., A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder, IEEE Robot. Autom. Lett. 3 (3) (2018) 1544–1551.
[13]
Zhao H., Wang Y., Duan J., Huang C., Cao D., Tong Y., Xu B., Bai J., Tong J., Zhang Q., Multivariate time-series anomaly detection via graph attention network, in: ICDM, IEEE, 2020, pp. 841–850.
[14]
Deng A., Hooi B., Graph neural network-based anomaly detection in multivariate time series, AAAI 35 (5) (2021) 4027–4035.
[15]
S. Han, S.S. Woo, Learning Sparse Latent Graph Representations for Anomaly Detection in Multivariate Time Series, in: KDD, 2022, pp. 2977–2986.
[16]
Little R.J., Rubin D.B., Statistical Analysis with Missing Data, Vol. 793, John Wiley & Sons, 2019.
[17]
Mitra R., McGough S.F., Chakraborti T., Holmes C., Copping R., Hagenbuch N., Biedermann S., Noonan J., Lehmann B., Shenvi A., et al., Learning from data with structured missingness, Nat. Mach. Intell. 5 (1) (2023) 13–23.
[18]
Beretta L., Santaniello A., Nearest neighbor imputation algorithms: a critical evaluation, BMC Med. Inf. Decis. Mak. 16 (3) (2016) 197–208.
[19]
Durbin J., Koopman S.J., Time Series Analysis By State Space Methods, Vol. 38, OUP Oxford, 2012.
[20]
Ahmad S., Lavin A., Purdy S., Agha Z., Unsupervised real-time anomaly detection for streaming data, Neurocomputing 262 (2017) 134–147.
[21]
Song H., Rajan D., Thiagarajan J., Spanias A., Attend and diagnose: Clinical time series analysis using attention models, AAAI 32 (1) (2018).
[22]
Chen Z., Chen D., Zhang X., Yuan Z., Cheng X., Learning graph structures with transformer for multivariate time-series anomaly detection in IoT, IEEE IoT J. 9 (12) (2021) 9179–9189.
[23]
Zhang C., Song D., Chen Y., Feng X., Lumezanu C., Cheng W., Ni J., Zong B., Chen H., Chawla N.V., A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data, AAAI 33 (01) (2019) 1409–1416.
[24]
Dai E., Chen J., Graph-augmented normalizing flows for anomaly detection of multiple time series, ICLR (2022).
[25]
M.M. Breunig, H.-P. Kriegel, R.T. Ng, J. Sander, LOF: identifying density-based local outliers, in: KDD, 2000, pp. 93–104.
[26]
Ting K.M., Liu Z., Zhang H., Zhu Y., A new distributional treatment for time series and an anomaly detection investigation, Proc. VLDB Endow. 15 (11) (2022) 2321–2333.
[27]
L. Shen, Z. Li, J. Kwok, Timeseries anomaly detection using temporal hierarchical one-class network, in: NeurIPS, Vol. 33, 2020, pp. 13016–13026.
[28]
Y. Shin, S. Lee, S. Tariq, M.S. Lee, O. Jung, D. Chung, S.S. Woo, ITAD: integrative tensor-based anomaly detection system for reducing false positives of satellite systems, in: CIKM, 2020, pp. 2733–2740.
[29]
Zhang H., Wu B., Yuan X., Pan S., Tong H., Pei J., Trustworthy graph neural networks: Aspects, methods and trends, 2022, arXiv preprint arXiv:2205.07424.
[30]
Zhang H., Wu B., Wang S., Yang X., Xue M., Pan S., Yuan X., Demystifying uneven vulnerability of link stealing attacks against graph neural networks, in: International Conference on Machine Learning, PMLR, 2023, pp. 41737–41752.
[31]
Zhang H., Yuan X., Nguyen Q.V.H., Pan S., On the interaction between node fairness and edge privacy in graph neural networks, 2023, arXiv preprint arXiv:2301.12951.
[32]
Koh H.Y., Nguyen A.T., Pan S., May L.T., Webb G.I., PSICHIC: physicochemical graph neural network for learning protein-ligand interaction fingerprints from sequence data, bioRxiv (2023) 1–26,.
[33]
Nguyen A.T.N., Nguyen D.T.N., Koh H.Y., Toskov J., MacLean W., Xu A., Zhang D., Webb G.I., May L.T., Halls M.L., The application of artificial intelligence to accelerate g protein-coupled receptor drug discovery, British J. Pharmacol. (2023).
[34]
Zheng Y., Koh H.Y., Ju J., Nguyen A.T., May L.T., Webb G.I., Pan S., Large language models for scientific synthesis, inference and explanation, 2023, arXiv preprint arXiv:2310.07984.
[35]
Koh H.Y., Ju J., Liu M., Pan S., An empirical survey on long document summarization: Datasets, models, and metrics, ACM Comput. Surv. 55 (8) (2022) 1–35.
[36]
H.Y. Koh, J. Ju, H. Zhang, M. Liu, S. Pan, How Far are We from Robust Long Abstractive Summarization?, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 2682–2698.
[37]
Seo Y., Defferrard M., Vandergheynst P., Bresson X., Structured sequence modeling with graph convolutional recurrent networks, in: ICONIP, Springer, 2018, pp. 362–373.
[38]
Jin M., Koh H.Y., Wen Q., Zambon D., Alippi C., Webb G.I., King I., Pan S., A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection, 2023, arXiv preprint arXiv:2307.03759.
[39]
Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, C. Zhang, Connecting the dots: Multivariate time series forecasting with graph neural networks, in: KDD, 2020, pp. 753–763.
[40]
Zheng Y., Koh H.Y., Jin M., Chi L., Phan K.T., Pan S., Chen Y.-P.P., Xiang W., Correlation-aware spatial–temporal graph learning for multivariate time-series anomaly detection, IEEE Trans. Neural Netw. Learn. Syst. (2023).
[41]
Jin M., Zheng Y., Li Y.-F., Chen S., Yang B., Pan S., Multivariate time series forecasting with dynamic graph neural ODEs, IEEE TKDE (2022).
[42]
Duan Z., Xu H., Wang Y., Huang Y., Ren A., Xu Z., Sun Y., Wang W., Multivariate time-series classification with hierarchical variational graph pooling, Neural Netw. 154 (2022) 481–490.
[43]
Rubin D.B., Inference and missing data, Biometrika 63 (3) (1976) 581–592.
[44]
Choi J., Choi H., Hwang J., Park N., Graph neural controlled differential equations for traffic forecasting, AAAI 36 (6) (2022) 6367–6374.
[45]
P. Kidger, J. Morrill, J. Foster, T. Lyons, Neural controlled differential equations for irregular time series, in: NeurIPS, Vol. 33, 2020, pp. 6696–6707.
[46]
Chen R.T., Rubanova Y., Bettencourt J., Duvenaud D.K., Neural ordinary differential equations, NeurIPS 31 (2018).
[47]
A. Sankar, Y. Wu, L. Gou, W. Zhang, H. Yang, DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks, in: WSDM, 2020, pp. 519–527.
[48]
McKinley S., Levine M., Cubic spline interpolation, College Redwoods 45 (1) (1998) 1049–1060.
[49]
Kipf T.N., Welling M., Semi-supervised classification with graph convolutional networks, ICLR (2017).
[50]
C.M. Ahmed, V.R. Palleti, A.P. Mathur, WADI: a water distribution testbed for research in the design of secure cyber physical systems, in: CySWater, 2017, pp. 25–28.
[51]
Zong B., Song Q., Min M.R., Cheng W., Lumezanu C., Cho D., Chen H., Deep autoencoding gaussian mixture model for unsupervised anomaly detection, ICLR (2018).
[52]
Tipping M.E., Bishop C.M., Probabilistic principal component analysis, J. R. Stat. Soc. Ser. B Stat. Methodol. 61 (3) (1999) 611–622.
[53]
Rousseeuw P.J., Silhouettes: a graphical aid to the interpretation and validation of cluster analysis, J. Comput. Appl. Math. 20 (1987) 53–65.
[54]
Behrmann J., Grathwohl W., Chen R.T., Duvenaud D., Jacobsen J.-H., Invertible residual networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 573–582.
[55]
Webb G.I., Hyde R., Cao H., Nguyen H.L., Petitjean F., Characterizing concept drift, DMKD 30 (4) (2016) 964–994.
[56]
Goldenberg I., Webb G.I., PCA-based drift and shift quantification framework for multidimensional data, Knowl. Inf. Syst. 62 (7) (2020) 2835–2854.
[57]
Pan S., Zheng Y., Liu Y., Integrating graphs with large language models: Methods and prospects, 2023, arXiv preprint arXiv:2310.05499.
[58]
Pan S., Luo L., Wang Y., Chen C., Wang J., Wu X., Unifying large language models and knowledge graphs: A roadmap, 2023, arXiv preprint arXiv:2306.08302.
[59]
Luo L., Li Y.-F., Haffari G., Pan S., Reasoning on graphs: Faithful and interpretable large language model reasoning, 2023, arXiv preprint arXiv:2310.01061.
[60]
Jin M., Wang S., Ma L., Chu Z., Zhang J.Y., Shi X., Chen P.-Y., Liang Y., Li Y.-F., Pan S., et al., Time-llm: Time series forecasting by reprogramming large language models, 2023, arXiv preprint arXiv:2310.01728.
[61]
Jin M., Wen Q., Liang Y., Zhang C., Xue S., Wang X., Zhang J., Wang Y., Chen H., Li X., et al., Large models for time series and spatio-temporal data: A survey and outlook, 2023, arXiv preprint arXiv:2310.10196.

Cited By

View all
  • (2024)Disentangled Anomaly Detection For Multivariate Time SeriesCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651492(931-934)Online publication date: 13-May-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Information Fusion
Information Fusion  Volume 106, Issue C
Jun 2024
580 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 25 June 2024

Author Tags

  1. Time series
  2. Anomaly forecasting
  3. Graph neural networks

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Disentangled Anomaly Detection For Multivariate Time SeriesCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651492(931-934)Online publication date: 13-May-2024

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media