Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Towards Faster Deep Graph Clustering via Efficient Graph Auto-Encoder

Published: 16 August 2024 Publication History

Abstract

Deep graph clustering (DGC) has been a promising method for clustering graph data in recent years. However, existing research primarily focuses on optimizing clustering outcomes by improving the quality of embedded representations, resulting in slow-speed complex models. Additionally, these methods do not consider changes in node similarity and corresponding adjustments in the original structure during the iterative optimization process after updating node embeddings, which easily falls into the representation collapse issue. We introduce an Efficient Graph Auto-Encoder (EGAE) and a dynamic graph weight updating strategy to address these issues, forming the basis for our proposed Fast DGC (FastDGC) network. Specifically, we significantly reduce feature dimensions using a linear transformation that preserves the original node similarity. We then employ a single-layer graph convolutional filtering approximation to replace multiple layers of graph convolutional neural network, reducing computational complexity and parameter count. During iteration, we calculate the similarity between nodes using the linearly transformed features and periodically update the original graph structure to reduce edges with low similarity, thereby enhancing the learning of discriminative and cohesive representations. Theoretical analysis confirms that EGAE has lower computational complexity. Extensive experiments on standard datasets demonstrate that our proposed method improves clustering performance and achieves a speedup of 2–3 orders of magnitude compared to state-of-the-art methods, showcasing outstanding performance. The code for our model is available at https://github.com/Marigoldwu/FastDGC. Furthermore, we have organized a portion of the DGC code into a unified framework, available at https://github.com/Marigoldwu/A-Unified-Framework-for-Deep-Attribute-Graph-Clustering.

Supplementary Material

TKDD-2023-11-0800-SUPP (tkdd-2023-11-0800-supp.zip)
Supplementary material

References

[1]
Amit Aflalo, Shai Bagon, Tamar Kashti, and Yonina C. Eldar. 2023. DeepCut: Unsupervised segmentation using graph neural networks clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 32–41. DOI:
[2]
Kamal Berahmand, Yuefeng Li, and Yue Xu. 2023a. DAC-HPP: Deep attributed clustering with high-order proximity preserve. Neural Computing and Applications 35, 34 (2023), 24493–24511. DOI:
[3]
Kamal Berahmand, Yuefeng Li, and Yue Xu. 2023b. A Deep semi-supervised community detection based on point-wise mutual information. IEEE Transactions on Computational Social Systems 11, 3 (2023), 3444–3456. DOI:
[4]
Kamal Berahmand, Mehrnoush Mohammadi, Razieh Sheikhpour, Yuefeng Li, and Yue Xu. 2024. WSNMF: Weighted symmetric nonnegative matrix factorization for attributed graph clustering. Neurocomputing 566 (2024), Article 127041. DOI:
[5]
Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, and Peng Cui. 2020. Structural deep clustering network. In Proceedings of the Web Conference 2020. ACM, New York, NY, 1400–1410. DOI:
[6]
Aleksandar Bojchevski and Stephan Günnemann. 2018. Deep Gaussian embedding of graphs: Unsupervised inductive learning via ranking. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=r1ZdKJ-0W
[7]
Sandro Cavallari, Vincent W. Zheng, Hongyun Cai, Kevin Chen-Chuan Chang, and Erik Cambria. 2017. Learning community embedding with community detection and node embedding on graphs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, New York, NY, 377–386. DOI:
[8]
Ganqu Cui, Jie Zhou, Cheng Yang, and Zhiyuan Liu. 2020. Adaptive graph encoder for attributed graph embedding. In Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 976–985. DOI:
[9]
Shifei Ding, Benyu Wu, Xiao Xu, Lili Guo, and Ling Ding. 2023. Graph clustering network with structure embedding enhanced. Pattern Recognition 144 (2023), Article 109833. DOI:
[10]
Lei Gong, Wenxuan Tu, Sihang Zhou, Long Zhao, Zhe Liu, and Xinwang Liu. 2022a. Deep fusion clustering network with reliable structure preservation. IEEE Transactions on Neural Networks and Learning Systems 35, 6 (2022), 7792–7803. DOI:
[11]
Lei Gong, Sihang Zhou, Wenxuan Tu, and Xinwang Liu. 2022b. Attributed graph clustering with dual redundancy reduction. In Proceedings of the 31st International Joint Conference on Artificial Intelligence. IJCAI, 3015–3021. DOI:
[12]
Lin Guo and Qun Dai. 2022. Graph clustering via variational graph embedding. Pattern Recognition 122 (2022), Article 108334.
[13]
Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. 2017. Improved deep embedded clustering with local structure preservation. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. IJCAI, 1753–1759. DOI:
[14]
Jie Hao and William Zhu. 2023. Deep graph clustering with enhanced feature representations for community detection. Applied Intelligence 53, 2 (2023), 1336–1349. DOI:
[15]
Kaveh Hassani and Amir Hosein Khas Ahmadi. 2020. Contrastive multi-view representation learning on graphs. In Proceedings of the 37th International Conference on Machine Learning, Vol. 119. PMLR, 4116–4126. http://proceedings.mlr.press/v119/hassani20a.html
[16]
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. 2022. GraphMAE: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 594–604. DOI:
[17]
Guangyu Huo, Yong Zhang, Junbin Gao, Boyue Wang, Yongli Hu, and Baocai Yin. 2023. CaEGCN: Cross-attention fusion based enhanced graph convolutional network for clustering. IEEE Transactions on Knowledge & Data Engineering 35, 4 (2023), 3471–3483. DOI:
[18]
Anil K. Jain, M Narasimha Murty, and Patrick J. Flynn. 1999. Data clustering: A review. ACM computing Surveys (CSUR) 31, 3 (1999), 264–323.
[19]
Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv:1611.07308. Retrieved from https://arxiv.org/abs/1611.07308
[20]
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations. https://openreview.net/forum?id=SJU4ayYgl
[21]
Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. 2023. What’s behind the mask: Understanding masked graph modeling for graph autoencoders. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 1268–1279. DOI:
[22]
Xunkai Li, Youpeng Hu, Yaoqi Sun, Ji Hu, Jiyong Zhang, and Meixia Qu. 2020. A deep graph structured clustering network. IEEE Access 8 (2020), 161727–161738. DOI:
[23]
Xuelong Li, Hongyuan Zhang, and Rui Zhang. 2021. Adaptive graph auto-encoder for general data clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 12 (2021), 9725–9732. DOI:
[24]
Yue Liu, Wenxuan Tu, Sihang Zhou, Xinwang Liu, Linxuan Song, Xihong Yang, and En Zhu. 2022a. Deep graph clustering via dual correlation reduction. In Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI Press, 7603–7611. DOI:
[25]
Yue Liu, Jun Xia, Sihang Zhou, Xihong Yang, Ke Liang, Chenchen Fan, Yan Zhuang, Stan Z Li, Xinwang Liu, and Kunlun He. 2022b. A survey of deep graph clustering: Taxonomy, challenge, application, and open resource. arXiv:2211.12875. Retrieved from https://arxiv.org/abs/2211.12875
[26]
Yue Liu, Xihong Yang, Sihang Zhou, Xinwang Liu, Zhen Wang, Ke Liang, Wenxuan Tu, Liang Li, Jingcan Duan, and Cancan Chen. 2023. Hard sample aware network for contrastive deep graph clustering. In Proceedings of the 37th AAAI Conference on Artificial Intelligence. AAAI Press, 8914–8922. DOI:
[27]
Amir Markovitz, Gilad Sharir, Itamar Friedman, Lihi Zelnik-Manor, and Shai Avidan. 2020. Graph embedded pose clustering for anomaly detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 10536–10544. DOI:
[28]
Nairouz Mrabah, Mohamed Bouguessa, Mohamed Fawzi Touati, and Riadh Ksantini. 2023. Rethinking graph auto-encoder models for attributed graph clustering. IEEE Trans. Knowl. Data Eng. 35, 9 (2023), 9037–9053. DOI:
[29]
James Munkres. 1957. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics 5, 1 (1957), 32–38.
[30]
Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. 2018. Adversarially regularized graph autoencoder for graph embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. IJCAI, 2609–2615. DOI:
[31]
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-GCN: Geometric graph convolutional networks. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=S1e2agrFvS
[32]
Zhihao Peng, Hui Liu, Yuheng Jia, and Junhui Hou. 2021. Attention-driven graph clustering network. In Proceedings of the ACM Multimedia Conference. ACM, New York, NY, 935–943. DOI:
[33]
Amin Salehi and Hasan Davulcu. 2020. Graph attention auto-encoders. In Proceedings of the 32nd IEEE International Conference on Tools with Artificial Intelligence. IEEE, 989–996. DOI:
[34]
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation. arXiv:1811.05868. Retrieved from https://arxiv.org/abs/1811.05868
[35]
Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo-Hyun Choi, Li Li, Rui Chen, and Xia Hu. 2023. S2GAE: Self-supervised graph autoencoders are generalizable learners with graph masking. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining. ACM, New York, NY, 787–795. DOI:
[36]
Mingyue Tang, Pan Li, and Carl Yang. 2022. Graph auto-encoder via neighborhood wasserstein reconstruction. In Proceedings of the 10th International Conference on Learning Representations. https://openreview.net/forum?id=ATUh28lnSuW
[37]
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. 2020. What makes for good views for contrastive learning? Advances in Neural Information Processing Systems 33 (2020), 6827–6839. https://proceedings.neurips.cc/paper/2020/hash/4c2e5eaae9152079b9e95845750bb9ab-Abstract.html
[38]
Wenxuan Tu, Sihang Zhou, Xinwang Liu, Xifeng Guo, Zhiping Cai, En Zhu, and Jieren Cheng. 2021. Deep fusion clustering network. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. AAAI Press, 9978–9987. DOI:
[39]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008), 2579–2605. https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf?fbcl
[40]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations. https://openreview.net/forum?id=rJXMpikCZ
[41]
Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Attributed graph clustering: A deep attentional embedding approach. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. IJCAI, 3670–3676. DOI:
[42]
Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. 2017. MGAE: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, New York, NY, 889–898. DOI:
[43]
Bernard L Welch. 1947. The generalization of ‘STUDENT’S’ problem when several different population variances are involved. Biometrika 34, 1–2 (1947), 28–35.
[44]
Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Q. Weinberger. 2019. Simplifying graph convolutional networks. In Proceedings of the 36th International Conference on Machine Learning, Vol. 97. PMLR, 6861–6871. http://proceedings.mlr.press/v97/wu19e.html
[45]
Junyuan Xie, Ross B. Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In Proceedings of the 33nd International Conference on Machine Learning (JMLR Workshop and Conference Proceedings, Vol. 48. JMLR, 478–487. http://proceedings.mlr.press/v48/xieb16.html
[46]
Rui Xu and Donald Wunsch. 2005. Survey of clustering algorithms. IEEE Transactions on Neural Networks 16, 3 (2005), 645–678. DOI:
[47]
Xiaoqiang Yan, Xiangyu Yu, Shizhe Hu, and Yangdong Ye. 2023. Mutual boost network for attributed graph clustering. Expert Systems with Applications 229 (2023), Article 120479. DOI:
[48]
Xihong Yang, Yue Liu, Sihang Zhou, Siwei Wang, Wenxuan Tu, Qun Zheng, Xinwang Liu, Liming Fang, and En Zhu. 2023. Cluster-guided contrastive graph clustering network. In Proceedings of the 37th AAAI Conference on Artificial Intelligence. AAAI Press, 10834–10842. DOI:
[49]
Yaowen Ye, Lianghao Xia, and Chao Huang. 2023. Graph masked autoencoder for sequential recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’23), 321–330. DOI:
[50]
Kaihua Zhang, Tengpeng Li, Shiwen Shen, Bo Liu, Jin Chen, and Qingshan Liu. 2020. Adaptive graph convolutional network with attention graph clustering for co-saliency detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 9047–9056. DOI:
[51]
Xiaotong Zhang, Han Liu, Qimai Li, and Xiao-Ming Wu. 2019. Attributed graph clustering via adaptive graph convolution. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. IJCAI, 4327–4333. DOI:
[52]
Han Zhao, Xu Yang, Zhenru Wang, Erkun Yang, and Cheng Deng. 2021b. Graph debiased contrastive learning with joint representation clustering. In Proceedings of the 30th International Joint Conference on Artificial Intelligence. IJCAI, 3434–3440. DOI:
[53]
Tong Zhao, Yozen Liu, Leonardo Neves, Oliver J. Woodford, Meng Jiang, and Neil Shah. 2021a. Data augmentation for graph neural networks. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. AAAI Press, 11015–11023. DOI:

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data
ACM Transactions on Knowledge Discovery from Data  Volume 18, Issue 8
September 2024
700 pages
EISSN:1556-472X
DOI:10.1145/3613713
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 August 2024
Online AM: 28 June 2024
Accepted: 22 June 2024
Revised: 23 April 2024
Received: 23 November 2023
Published in TKDD Volume 18, Issue 8

Check for updates

Author Tags

  1. Deep graph clustering
  2. graph auto-encoder
  3. graph neural networks
  4. unsupervised learning

Qualifiers

  • Research-article

Funding Sources

  • National Natural Science Foundation of China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 287
    Total Downloads
  • Downloads (Last 12 months)287
  • Downloads (Last 6 weeks)105
Reflects downloads up to 16 Oct 2024

Other Metrics

Citations

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media