Abstract
Graph anomaly detection, aimed at identifying anomalous patterns that significantly differ from other nodes, has drawn widespread attention in recent years. Due to the complex topological structures and attribute information inherent in graphs, conventional methods often struggle to effectively identify anomalies. Deep anomaly detection methods based on Graph Neural Networks (GNNs) have achieved significant success. However, they face the challenge of not only obtaining limited neighborhood information but over-smoothing. Over-smoothing is the phenomenon where the representations of nodes gradually become similar and flattened across multiple convolutional layers, thereby limiting the comprehensive learning of neighborhood information. Therefore, we propose a novel anomaly detection framework, TransGAD, to address these challenges. Inspired by the Graph Transformer, we introduce a Transformer-based autoencoder. Treating each node as a sequence and its neighborhood as tokens in the sequence, this autoencoder captures both local and global information. We incorporate cosine positional encoding and masking strategy to obtain more informative node representations and leverage reconstruction error for improved anomaly detection. Experimental results on seven datasets demonstrate that our approach outperforms the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Branco, B., Abreu, P., Gomes, A.S., Almeida, M.S., Ascensão, J.T., Bizarro, P.: Interleaved sequence RNNs for fraud detection. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 3101–3109 (2020)
Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., Sun, X.: Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3438–3445 (2020)
Chen, J., Gao, K., Li, G., He, K.: Nagphormer: a tokenized graph transformer for node classification in large graphs. In: The Eleventh International Conference on Learning Representations (2022)
Ding, K., Li, J., Bhanushali, R., Liu, H.: Deep anomaly detection on attributed networks. In: Proceedings of the 2019 SIAM International Conference on Data Mining, pp. 594–602. SIAM (2019)
Duan, J., et al.: Graph anomaly detection via multi-scale contrastive learning networks with augmented view. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 37, pp. 7459–7467 (2023)
Fan, H., Zhang, F., Li, Z.: Anomalydae: dual autoencoder for anomaly detection on attributed networks. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5685–5689. IEEE (2020)
Kim, H., Lee, B.S., Shin, W.Y., Lim, S.: Graph anomaly detection with graph neural networks: Current status and challenges. IEEE Access (2022)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kreuzer, D., Beaini, D., Hamilton, W., Létourneau, V., Tossou, P.: Rethinking graph transformers with spectral attention. Adv. Neural. Inf. Process. Syst. 34, 21618–21629 (2021)
Li, J., Dani, H., Hu, X., Liu, H.: Radar: residual analysis for anomaly detection in attributed networks. In: IJCAI, vol. 17, pp. 2152–2158 (2017)
Liu, K., et al.: Bond: benchmarking unsupervised outlier node detection on static attributed graphs. Adv. Neural. Inf. Process. Syst. 35, 27021–27035 (2022)
Liu, Y., Li, Z., Pan, S., Gong, C., Zhou, C., Karypis, G.: Anomaly detection on attributed networks via contrastive self-supervised learning. IEEE Trans. Neural Netw. Learn. Syst. 33(6), 2378–2392 (2021)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
Luo, X., et al.: Comga: community-aware attributed graph anomaly detection. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 657–665 (2022)
Ma, X., et al.: A comprehensive survey on graph anomaly detection with deep learning. IEEE Transactions on Knowledge and Data Engineering (2021)
Miao, K., Shi, X., Zhang, W.A.: Attack signal estimation for intrusion detection in industrial control system. Comput. Security 96, 101926 (2020)
Min, E., et al.: Transformer for graphs: an overview from architecture perspective. arXiv preprint arXiv:2202.08455 (2022)
Peng, Z., Luo, M., Li, J., Liu, H., Zheng, Q., et al.: Anomalous: a joint modeling approach for anomaly detection on attributed networks. In: IJCAI, pp. 3513–3519 (2018)
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93–93 (2008)
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., Su, Z.: Arnetminer: extraction and mining of academic social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 990–998 (2008)
Tang, L., Liu, H.: Relational learning via latent social dimensions. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 817–826 (2009)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)
Wu, N., Chen, F., Li, J., Huai, J., Zhou, B., Ramakrishnan, N., et al.: A nonparametric approach to uncovering connected anomalies by tree shaped priors. IEEE Trans. Knowl. Data Eng. 31(10), 1849–1862 (2018)
Wu, Z., Jain, P., Wright, M., Mirhoseini, A., Gonzalez, J.E., Stoica, I.: Representing long-range context for graph neural networks with global attention. Adv. Neural. Inf. Process. Syst. 34, 13266–13279 (2021)
Xiong, R., et al.: On layer normalization in the transformer architecture. In: International Conference on Machine Learning, pp. 10524–10533. PMLR (2020)
Xu, H., et al.: Beyond outlier detection: Outlier interpretation by attention-guided triplet deviation network. In: Proceedings of the Web Conference 2021, pp. 1328–1339 (2021)
Ye, J., Akoglu, L.: Discovering opinion spammer groups by network footprints. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part I 15. pp. 267–282. Springer (2015). https://doi.org/10.1007/978-3-319-23528-8_17
Ying, C., et al.: Do transformers really perform badly for graph representation? Adv. Neural. Inf. Process. Syst. 34, 28877–28888 (2021)
Yuan, X., Zhou, N., Yu, S., Huang, H., Chen, Z., Xia, F.: Higher-order structure based anomaly detection on attributed networks. In: 2021 IEEE International Conference on Big Data (Big Data), pp. 2691–2700. IEEE (2021)
Zhang, J., Wang, S., Chen, S.: Reconstruction enhanced multi-view contrastive learning for anomaly detection on attributed networks. arXiv preprint arXiv:2205.04816 (2022)
Zheng, Y., Jin, M., Liu, Y., Chi, L., Phan, K.T., Chen, Y.P.P.: Generative and contrastive self-supervised learning for graph anomaly detection. IEEE Transactions on Knowledge and Data Engineering (2021)
Acknowledgements
This work was supported by National Key Research and Development Program of China(31400), and partly supported by Natural Science Foundation of Inner Mongolia Autonomous Region of China (2022LHMS06008), Key R&D and Transformation Plan of Qinghai Province(2022-QY-218), and University-Industry Collaborative Education Program(202312280037).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Guo, Z., Wu, N., Zhao, Y., Wang, W. (2024). TransGAD: A Transformer-Based Autoencoder for Graph Anomaly Detection. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14855. Springer, Singapore. https://doi.org/10.1007/978-981-97-5572-1_17
Download citation
DOI: https://doi.org/10.1007/978-981-97-5572-1_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5571-4
Online ISBN: 978-981-97-5572-1
eBook Packages: Computer ScienceComputer Science (R0)