research-article

Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Authors:

Yi-Ping Phoebe ChenAuthors Info & Claims

IEEE Transactions on Knowledge and Data Engineering, Volume 35, Issue 12

Pages 12220 - 12233

https://doi.org/10.1109/TKDE.2021.3119326

Published: 12 October 2021 Publication History

Abstract

Anomaly detection from graph data has drawn much attention due to its practical significance in many critical applications including cybersecurity, finance, and social networks. Existing data mining and machine learning methods are either shallow methods that could not effectively capture the complex interdependency of graph data or graph autoencoder methods that could not fully exploit the contextual information as supervision signals for effective anomaly detection. To overcome these challenges, in this paper, we propose a novel method, Self-Supervised Learning for Graph Anomaly Detection (<monospace>SL-GAD</monospace>). Our method constructs different contextual subgraphs (views) based on a target node and employs two modules, <italic>generative attribute regression</italic> and <italic>multi-view contrastive learning</italic> for anomaly detection. While the <italic>generative attribute regression</italic>module allows us to capture the anomalies in the attribute space, the <italic>multi-view contrastive learning</italic> module can exploit richer structure information from multiple subgraphs, thus abling to capture the anomalies in the structure space, mixing of structure, and attribute information. We conduct extensive experiments on six benchmark datasets and the results demonstrate that our method outperforms state-of-the-art methods by a large margin.

References

[1]

T. Pourhabibi, K.-L. Ong, B. H. Kam, and Y. L. Boo, “Fraud detection: A systematic literature review of graph-based anomaly detection approaches,” Decis. Support Syst., vol. 133, 2020, Art. no.

Digital Library

[2]

M. Latah, “Detection of malicious social bots: A survey and a refined taxonomy,” Exp. Syst. Appl., vol. 151, 2020, Art. no.

[3]

B. Perozzi and L. Akoglu, “Scalable anomaly ranking of attributed neighborhoods,” in Proc. SIAM Int. Conf. Data Mining, 2016, pp. 207–215.

[4]

J. Li, H. Dani, X. Hu, and H. Liu, “Radar: Residual analysis for anomaly detection in attributed networks,” in Proc. Int. Joint Conf. Artif. Intell., 2017, pp. 2152–2158.

[5]

Z. Peng, M. Luo, J. Li, H. Liu, and Q. Zheng, “Anomalous: A joint modeling approach for anomaly detection on attributed networks,” in Proc. Int Joint Conf. Artif. Intell., 2018, pp. 3513–3519.

[6]

K. Ding, J. Li, R. Bhanushali, and H. Liu, “Deep anomaly detection on attributed networks,” in Proc. SIAM Int. Conf. Data Mining, 2019, pp. 594–602.

[7]

Y. Li, X. Huang, J. Li, M. Du, and N. Zou, “SpecAE: Spectral autoencoder for anomaly detection in attributed networks,” in Proc. 28th ACM Int. Conf. Inf. Know. Manage., 2019, pp. 2233–2236.

[8]

K. Ding, J. Li, N. Agarwal, and H. Liu, “Inductive anomaly detection on attributed networks,” in Proc. 29th Int. Joint Conf. Artif. Intell., 2020, pp. 1288–1294.

[9]

V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surv., vol. 41, no. 3, pp. 1–58, 2009.

Digital Library

[10]

G. Pang, C. Shen, L. Cao, and A. v. d. Hengel, “Deep learning for anomaly detection: A review,” 2020,.

[11]

M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying density-based local outliers,” in Proc. ACM SIGMOD Int. Conf. Manage. Data, 2000, pp. 93–104.

[12]

B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, “Estimating the support of a high-dimensional distribution,” Neural Comput., vol. 13, no. 7, pp. 1443–1471, 2001.

Digital Library

[13]

D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” 2018,.

[14]

L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft, “Deep semi-supervised anomaly detection,” 2019,.

[15]

Y. Liu, Z. Li, S. Pan, C. Gong, C. Zhou, and G. Karypis, “Anomaly detection on attributed networks via contrastive self-supervised learning,” IEEE Trans. Neural Netw. Learn. Syst., early access, Apr. 5, 2021.

[16]

X. Liu et al., “Self-supervised learning: Generative or contrastive,” 2020,.

[17]

L. Jing and Y. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 11, pp. 4037–4058, Nov. 2021.

[18]

Y. Liu, S. Pan, M. Jin, C. Zhou, F. Xia, and P. S. Yu, “Graph self-supervised learning: A survey,” 2021,.

[19]

P. Velickovic, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm, “Deep graph infomax,” Proc. Int. Conf. Learn. Representations (Poster), vol. 2, no. 3, p. 4, 2019.

[20]

K. Hassani and A. H. Khasahmadi, “Contrastive multi-view representation learning on graphs,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 4116–4126.

[21]

S. Wan, S. Pan, J. Yang, and C. Gong, “Contrastive and generative graph convolutional networks for graph-based semi-supervised learning,” in Proc. AAAI Conf. Artif. Intell., 2021, pp. 10049–10057.

[22]

M. Jin, Y. Zheng, Y.-F. Li, C. Gong, C. Zhou, and S. Pan, “Multi-scale contrastive siamese networks for self-supervised graph representation learning,” in Proc. Int. Joint Conf. Artif. Intell., 2021, pp. 1477–1483.

[23]

Y. You, T. Chen, Y. Shen, and Z. Wang, “Graph contrastive learning automated,” in Proc. Int. Conf. Mach. Learn., 2021, pp. 12121–12132.

[24]

Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 1, pp. 4–24, Jan. 2021.

[25]

T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. Int. Conf. Learn. Representations, 2017, pp. 1–14.

[26]

P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in Proc. Int. Conf. Learn. Representations, 2018.

[27]

W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Proc. Neural Inf. Process. Syst., 2017, pp. 1024–1034.

[28]

F. Frasca, E. Rossi, D. Eynard, B. Chamberlain, M. Bronstein, and F. Monti, “SIGN: Scalable inception graph neural networks,” in Proc. ICML Workshop Graph Representation Learn. Beyond, 2020.

[29]

S. Geisler, D. Zügner, and S. Günnemann, “Reliable graph neural networks via robust aggregation,” Proc. Neural Inf. Process. Syst., vol. 33, pp. 13272–13284, 2020.

[30]

H. Wang, C. Zhou, X. Chen, J. Wu, S. Pan, and J. Wang, “Graph stochastic neural networks for semi-supervised learning,” Proc. Neural Inf. Process. Syst., vol. 33, pp. 19839–19848, 2020.

[31]

S. Zhu, S. Pan, C. Zhou, J. Wu, Y. Cao, and B. Wang, “Graph geometry interaction learning,” Proc. Neural Inf. Process. Syst., vol. 33, pp. 7548–7558, 2020.

[32]

M. Wu, S. Pan, L. Du, and X. Zhu, “Learning graph neural networks with positive and unlabeled nodes,” ACM Trans. Knowl. Discov. Data, vol. 15, no. 6, pp. 1–25, Jun. 2021.

Digital Library

[33]

S. Zhu, C. Zhou, S. Pan, X. Zhu, and B. Wang, “Relation structure-aware heterogeneous graph neural network,” in Proc. IEEE Int. Conf. Data Mining, 2019, pp. 1534–1539.

[34]

Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting the dots: Multivariate time series forecasting with graph neural networks,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2020, pp. 753–763.

[35]

D. Jin, Z. Yu, P. Jiao, S. Pan, P. S. Yu, and W. Zhang, “A survey of community detection approaches: From statistical modeling to deep learning,” 2021,.

[36]

S. Wan, C. Gong, P. Zhong, S. Pan, G. Li, and J. Yang, “Hyperspectral image classification with context-aware dynamic graph convolutional network,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 1, pp. 597–612, Jan. 2021.

[37]

T. N. Kipf and M. Welling, “Variational graph auto-encoders,” 2016,.

[38]

W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, and J. Leskovec, “Strategies for pre-training graph neural networks,” in Proc. Int. Conf. Learn. Representations, 2020.

[39]

Y. You, T. Chen, Z. Wang, and Y. Shen, “When does self-supervision help graph convolutional networks?,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 10 871–10 880.

[40]

Y. Jiao, Y. Xiong, J. Zhang, Y. Zhang, T. Zhang, and Y. Zhu, “Sub-graph contrast for scalable self-supervised graph representation learning,” in Proc. IEEE Int. Conf. Data Mining, 2020, pp. 222–231.

[41]

Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in Proc. Eur. Conf. Comput. Vis. 2020, pp. 776–794.

[42]

C. Tosh, A. Krishnamurthy, and D. Hsu, “Contrastive learning, multi-view redundancy, and linear models,” in Proc. Algorithmic Learn. Theory, 2021, pp. 1179–1206.

[43]

H. Tong, C. Faloutsos, and J.-Y. Pan, “Fast random walk with restart and its applications,” in Proc. Int. Conf. Data Mining, 2006, pp. 613–622.

[44]

J. Klicpera, S. Weißenberger, and S. Günnemann, “Diffusion improves graph learning,” in Proc. Neural Inf. Process. Syst., 2019, pp. 13354–13366.

[45]

Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen, “Graph contrastive learning with augmentations,” Proc. Neural Inf. Process. Syst., vol. 33, pp. 5812–5823, 2020.

[46]

C. Zhou and R. C. Paffenroth, “Anomaly detection with robust deep autoencoders,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2017, pp. 665–674.

[47]

Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec, “Hierarchical graph representation learning with differentiable pooling,” in Proc. Neural Inf. Process. Syst., 2018, pp. 4800–4810.

[48]

L. Tang and H. Liu, “Relational learning via latent social dimensions,” in Proc. 15th ACM SIGKDD Int. Conf. Knowledge Discov. Data Mining, 2009, pp. 817–826.

[49]

J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su, “ArnetMiner: Extraction and mining of academic social networks,” in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2008, pp. 990–998.

[50]

P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad, “Collective classification in network data,” AI Mag., vol. 29, no. 3, pp. 93–93, 2008.

Digital Library

[51]

M. Zhang et al., “One-shot neural architecture search: Maximising diversity to overcome catastrophic forgetting,” IEEE Trans. Pattern Anal Mach. Intell., vol. 43, no. 9, pp. 2921–2935, Sep. 2021.

[52]

M. Zhang, H. Li, S. Pan, X. Chang, Z. Ge, and S. W. Su, “Differentiable neural architecture search in equivalent space with exploration enhancement,” in Proc. Neural Inf. Process. Syst., 2020.

[53]

M. Zhang, H. Li, S. Pan, X. Chang, and S. Su, “Overcoming multi-model forgetting in one-shot NAS with diversity maximization,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 7809–7818.

Cited By

Gong ZSun YBaeza-Yates RBonchi F(2024)An Energy-centric Framework for Category-free Out-of-distribution Node Detection in GraphsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671939(908-919)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671939
Ma XLi RLiu FDing KYang JWu JBaeza-Yates RBonchi F(2024)Graph Anomaly Detection with Few Labels: A Data-Centric ApproachProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671929(2153-2164)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671929
Pazho ANoghre GPurkayastha AVempati JMartin OTabkhi H(2024)A Survey of Graph-Based Deep Learning for Anomaly Detection in Distributed SystemsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.328289836:1(1-20)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3282898
Show More Cited By

Index Terms

Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection

Index terms have been assigned to the content through auto-classification.

Recommendations

SSCL: Semi-supervised Contrastive Learning for Industrial Anomaly Detection
Pattern Recognition and Computer Vision
Abstract
Anomaly detection is an important machine learning task that aims to identify data points that are inconsistent with normal data patterns. In real-world scenarios, it is common to have access to some labeled and unlabeled samples that are known to ...
Autoencoding Binary Classifiers for Supervised Anomaly Detection
PRICAI 2019: Trends in Artificial Intelligence
Abstract
We propose the Autoencoding Binary Classifiers (ABC), a novel supervised anomaly detector based on the Autoencoder (AE). There are two main approaches in anomaly detection: supervised and unsupervised. The supervised approach accurately detects ...
GIAD: Generative Inpainting-Based Anomaly Detection via Self-Supervised Learning for Human Monitoring
PRICAI 2021: Trends in Artificial Intelligence
Abstract
Detecting anomalies in human monitoring is an important task in many real-world applications. In addition to typical anomalies of objects or actions which have never been observed before, there are also anomalous combinations where we need to ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Knowledge and Data Engineering

IEEE Transactions on Knowledge and Data Engineering Volume 35, Issue 12

Dec. 2023

1114 pages

ISSN:1041-4347

Issue’s Table of Contents

1041-4347 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 12 October 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gong ZSun YBaeza-Yates RBonchi F(2024)An Energy-centric Framework for Category-free Out-of-distribution Node Detection in GraphsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671939(908-919)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671939
Ma XLi RLiu FDing KYang JWu JBaeza-Yates RBonchi F(2024)Graph Anomaly Detection with Few Labels: A Data-Centric ApproachProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671929(2153-2164)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671929
Pazho ANoghre GPurkayastha AVempati JMartin OTabkhi H(2024)A Survey of Graph-Based Deep Learning for Anomaly Detection in Distributed SystemsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.328289836:1(1-20)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3282898
Zhang HRen YFu LWang XChen GZhou C(2024)Multi-Scale Self-Supervised Graph Contrastive Learning With Injective Node AugmentationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.327846336:1(261-274)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3278463
Zeb SMahmood AKhowaja SDev KHassan SGidlund MBellavista P(2024)Towards defining industry 5.0 vision with intelligent and softwarized wireless network architectures and servicesJournal of Network and Computer Applications10.1016/j.jnca.2023.103796223:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.jnca.2023.103796
Zhang WPei ZWang F(2024)FeatsFlowEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107151126:PDOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107151
Parcollet TNguyen HEvain SZanon Boito MPupier AMdhaffar SLe HAlisamir STomashenko NDinarelli MZhang SAllauzen ACoavoux MEstève YRouvier MGoulian JLecouteux BPortet FRossato SRingeval FSchwab DBesacier L(2024)LeBenchmark 2.0Computer Speech and Language10.1016/j.csl.2024.10162286:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.csl.2024.101622
Zhao NHan BLi RSu JZhou C(2023)A Multivariate KPIs Anomaly Detection Framework With Dynamic Balancing Loss TrainingIEEE Transactions on Network and Service Management10.1109/TNSM.2022.322480320:2(1418-1429)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TNSM.2022.3224803
Zhou SHuang XLiu NZhou HChung FHuang L(2023)Improving Generalizability of Graph Anomaly Detection Models via Data AugmentationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.327177135:12(12721-12735)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3271771
Pham NAbuadbba AGao YPhan KChilamkurti N(2023)Binarizing Split Learning for Data Privacy Enhancement and Computation ReductionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.327439118(3088-3100)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIFS.2023.3274391
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents