research-article

Efficient Graph Convolution for Joint Node Representation Learning and Clustering

Authors:

Mohamed NadifAuthors Info & Claims

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Pages 289 - 297

https://doi.org/10.1145/3488560.3498533

Published: 15 February 2022 Publication History

Abstract

Attributed graphs are used to model a wide variety of real-world networks. Recent graph convolutional network-based representation learning methods have set state-of-the-art results on the clustering of attributed graphs. However, these approaches deal with clustering as a downstream task while better performances can be attained by incorporating the clustering objective into the representation learning process. In this paper, we propose, in a unified framework, an objective function taking into account both tasks simultaneously. Based on a variant of the simple graph convolutional network, our model does clustering by minimizing the difference between the convolved node representations and their reconstructed cluster representatives. We showcase the efficiency of the derived algorithm against state-of-the-art methods both in terms of clustering performance and computational cost on thede facto benchmark graph clustering datasets. We further demonstrate the usefulness of the proposed approach for graph visualization through generating embeddings that exhibit a clustering structure.

Supplementary Material

MP4 File (WSDM22-fp493.mp4)

This is a video presentation for the paper "Efficient Graph Convolution for Joint Node Representation Learning and Clustering", which has been accepted in WSDM'22.

Download
12.46 MB

References

[1]

Amr Ahmed, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J Smola. 2013. Distributed large-scale natural graph factorization. In Proceedings of the 22nd international conference on World Wide Web. 37--48.

Digital Library

[2]

Kais Allab, Lazhar Labiod, and Mohamed Nadif. 2016. A semi-NMF-PCA unified framework for data clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 1 (2016), 2--16.

Digital Library

[3]

Kais Allab, Lazhar Labiod, and Mohamed Nadif. 2018. Simultaneous spectral data embedding and clustering. IEEE transactions on neural networks and learning systems, Vol. 29, 12 (2018), 6396--6401.

[4]

Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE, Vol. 2, 1 (2015), 1--18.

[5]

Bryan Perozzi Anton Tsitsulin, John Palowitch and Emmanuel Müller. 2020. Graph Clustering with Graph Neural Networks. In Proceedings of the 16th International Workshop on Mining and Learning with Graphs (MLG) .

[6]

Yuki M. Asano, Christian Rupprecht, and Andrea Vedaldi. 2020. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations (ICLR) .

[7]

Mikhail Belkin and Partha Niyogi. 2003. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation, Vol. 15, 6 (2003), 1373--1396.

Digital Library

[8]

Ganqu Cui, Jie Zhou, Cheng Yang, and Zhiyuan Liu. 2020. Adaptive graph encoder for attributed graph embedding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 976--985.

Digital Library

[9]

Geert De Soete and J Douglas Carroll. 1994. K-means clustering in a low-dimensional Euclidean space. In New approaches in classification and data analysis. Springer, 212--219.

[10]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, Vol. 29 (2016), 3844--3852.

[11]

Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis. 2007. Weighted Graph Cuts without Eigenvectors A Multilevel Approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, 11 (2007), 1944--1957.

Digital Library

[12]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. In The World Wide Web Conference (San Francisco, CA, USA) (WWW '19). Association for Computing Machinery, New York, NY, USA, 417--426. https://doi.org/10.1145/3308558.3313488

Digital Library

[13]

Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier. 2020. Deep k-means: Jointly clustering with k-means and learning representations. Pattern Recognition Letters, Vol. 138 (2020), 185--192.

[14]

Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, and Heng Huang. 2017. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference on computer vision. 5736--5745.

[15]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining . 855--864.

Digital Library

[16]

Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, Vol. 53, 2 (2011), 217--288.

[17]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE Data Eng. Bull., Vol. 40, 3 (2017), 52--74.

[18]

Kurt Hornik, Ingo Feinerer, Martin Kober, and Christian Buchta. 2012. Spherical k-Means Clustering. Journal of Statistical Software, Articles, Vol. 50, 10 (2012), 1--22.

[19]

Yedid Hoshen. 2017. VAIN: Attentional Multi-agent Predictive Modeling. In Neural Information Processing Systems (NIPS). 2701--2711.

[20]

Md Rezaul Karim, Oya Beyan, Achille Zappa, Ivan G Costa, Dietrich Rebholz-Schuhmann, Michael Cochez, and Stefan Decker. 2020 a. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics (2020), 1--23.

[21]

Md Rezaul Karim, Michael Cochez, Achille Zappa, Ratnesh Sahay, Dietrich Rebholz-Schuhmann, Oya Beyan, and Stefan Decker. 2020 b. Convolutional Embedded Networks for Population Scale Clustering and Bio-ancestry Inferencing. IEEE/ACM Transactions on Computational Biology and Bioinformatics (2020).

[22]

Thomas N Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. NIPS Workshop on Bayesian Deep Learning (2016).

[23]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations (ICLR) .

[24]

Lazhar Labiod and Mohamed Nadif. 2021. Efficient regularized spectral data embedding. Advances in Data Analysis and Classification, Vol. 15, 1 (2021), 99--119.

Digital Library

[25]

S. P. Lloyd. 1982. Least squares quantization in PCM. IEEE Trans. Inf. Theory, Vol. 28 (1982), 129--136.

Digital Library

[26]

Diego Marcheggiani and Ivan Titov. 2017. Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, 1506--1515.

[27]

Costas Mavromatis and George Karypis. 2021. Graph InfoClust: Maximizing Coarse-Grain Mutual Information in Graphs. In PAKDD (1) . 541--553.

[28]

Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2001. On Spectral Clustering: Analysis and an Algorithm. In International Conference on Neural Information Processing Systems: Natural and Synthetic. 849--856.

Digital Library

[29]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: online learning of social representations. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 701--710.

Digital Library

[30]

Xiaojuan Qi, Renjie Liao, Jiaya Jia, Sanja Fidler, and Raquel Urtasun. 2017. 3D Graph Neural Networks for RGBD Semantic Segmentation. In 2017 IEEE International Conference on Computer Vision (ICCV). 5209--5218.

[31]

Benedek Rozemberczki, Ryan Davies, Rik Sarkar, and Charles Sutton. 2019. Gemsec: Graph embedding with self clustering. In Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining . 65--72.

Digital Library

[32]

Aghiles Salah and Mohamed Nadif. 2017. Social regularized von Mises--Fisher mixture model for item recommendation. Data Mining and Knowledge Discovery, Vol. 31, 5 (2017), 1218--1241.

Digital Library

[33]

Guillaume Salha, Romain Hennequin, and Michalis Vazirgiannis. 2020. Simple and Effective Graph Autoencoders with One-Hop Linear Models. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD). 319--334.

[34]

Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. 2018. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning. 4470--4479.

[35]

Victor Garcia Satorras and Joan Bruna Estrach. 2018. Few-Shot Learning with Graph Neural Networks. In International Conference on Learning Representations . https://openreview.net/forum?id=BJj6qGbRW

[36]

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, Vol. 29, 3 (2008), 93--93.

[37]

Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. A Graph-to-Sequence Model for AMR-to-Text Generation. In the Association for Computational Linguistics, ACL 2018. 1616--1626.

[38]

Fan-Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In International Conference on Learning Representations . https://openreview.net/forum?id=r1lfF2NYvH

[39]

Laurens van der Maaten and Geoffrey E. Hinton. 2008. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research, Vol. 9 (2008), 2579--2605.

[40]

Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. ICLR (Poster), Vol. 2, 3 (2019), 4.

[41]

Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining . 1225--1234.

Digital Library

[42]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International conference on machine learning . 6861--6871.

[43]

Michio Yamamoto and Heungsun Hwang. 2014. A general formulation of cluster analysis with dimension reduction and subspace separation. Behaviormetrika, Vol. 41, 1 (2014), 115--129.

[44]

Bo Yang, Xiao Fu, Nicholas D. Sidiropoulos, and Mingyi Hong. 2017. Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). 3861--3870.

[45]

Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y. Chang. 2015. Network Representation Learning with Rich Text Information. In IJCAI .

Digital Library

[46]

Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, and Devi Parikh. 2018. Graph R-CNN for Scene Graph Generation., bibinfonumpages690--706 pages.

[47]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 974--983.

Digital Library

[48]

Xiaotong Zhang, Han Liu, Qimai Li, and Xiao-Ming Wu. 2019. Attributed Graph Clustering via Adaptive Graph Convolution. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 4327--4333.

[49]

Hao Zhu and Piotr Koniusz. 2021. Simple Spectral Graph Convolution. In 9th International Conference on Learning Representations, ICLR, Virtual Event, Austria, May 3--7, 2021 . OpenReview.net.

[50]

Wenwu Zhu, Xin Wang, and Peng Cui. 2020. Deep learning for learning graph representations. In Deep Learning: Concepts and Architectures. Springer, 169--210.

Cited By

Wang YYan XHu CXu QYang CFu FZhang WWang HDu BJiang J(2024)Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00234(3364-3378)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00234
Cheng JHe CHan KChen GLiang WTang Y(2024)Unveiling community structures in static networks through graph variational Bayes with evolution informationNeurocomputing10.1016/j.neucom.2024.127349576:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127349
Cao JXu WJin DZhang XLiu LMiller AShi ZDing W(2024)A unified framework of semi-supervised community detection integrating network topology and node contentInformation Sciences10.1016/j.ins.2024.121349(121349)Online publication date: Aug-2024
https://doi.org/10.1016/j.ins.2024.121349
Show More Cited By

Index Terms

Efficient Graph Convolution for Joint Node Representation Learning and Clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
        Dimensionality reduction and manifold learning

Recommendations

Multi-scale graph attention subspace clustering network
Abstract
Deep subspace clustering (DSC) network which uses deep auto-encoders (DAE) mapping raw data into a latent space for clustering has achieved significant performance. However, when it comes to graph-based clustering, it fails to encode the node ...
Simple and Effective Graph Autoencoders with One-Hop Linear Models
Machine Learning and Knowledge Discovery in Databases
Abstract
Over the last few years, graph autoencoders (AE) and variational autoencoders (VAE) emerged as powerful node embedding methods, with promising performances on challenging tasks such as link prediction and node clustering. Graph AE, VAE and most of ...
Deep neighbor-aware embedding for node clustering in attributed graphs
Highlights
- We proposed a neighbor-aware embedding algorithm for attributed graph clustering.
Abstract
Node clustering aims to partition the vertices in a graph into multiple groups or communities. Existing studies have mostly focused on developing deep learning approaches to learn a latent representation of nodes, based on which simple ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

February 2022

1690 pages

ISBN:9781450391320

DOI:10.1145/3488560

General Chairs:
K. Selcuk Candan
Arizona State University, USA
,
Huan Liu
Arizona State University, USA
,
Program Chairs:
Leman Akoglu
Carnegie Mellon University, USA
,
Xin Luna Dong
Meta Platforms, Inc. (former Facebook), USA
,
Jiliang Tang
Michigan State University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '22

Sponsor:

WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining

February 21 - 25, 2022

AZ, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,208
Total Downloads

Downloads (Last 12 months)275
Downloads (Last 6 weeks)19

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YYan XHu CXu QYang CFu FZhang WWang HDu BJiang J(2024)Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00234(3364-3378)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00234
Cheng JHe CHan KChen GLiang WTang Y(2024)Unveiling community structures in static networks through graph variational Bayes with evolution informationNeurocomputing10.1016/j.neucom.2024.127349576:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127349
Cao JXu WJin DZhang XLiu LMiller AShi ZDing W(2024)A unified framework of semi-supervised community detection integrating network topology and node contentInformation Sciences10.1016/j.ins.2024.121349(121349)Online publication date: Aug-2024
https://doi.org/10.1016/j.ins.2024.121349
Wang ZHe ZLiu D(2024)Graph analysis using a GPU-based parallel algorithm: quantum clusteringApplied Intelligence10.1007/s10489-024-05587-854:17-18(7765-7776)Online publication date: 14-Jun-2024
https://doi.org/10.1007/s10489-024-05587-8
Sun LWang FYe JPeng HYu PElkind E(2023)CONGREGATEProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/255(2296-2305)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/255
Gu MYang GZhou SMa NChen JTan QLiu MBu JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Homophily-enhanced Structure Learning for Graph ClusteringProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614915(577-586)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614915
Bi WDu LFu QWang YHan SZhang DChua TLauw HSi LTerzi ETsaparas P(2023)MM-GNN: Mix-Moment Graph Neural Network towards Modeling Neighborhood Feature DistributionProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570457(132-140)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3539597.3570457
Fettal CLabiod LNadif M(2023)Boosting Subspace Co-Clustering via Bilateral Graph ConvolutionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.330081436:3(960-971)Online publication date: 3-Aug-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3300814
Tan QZhang XHuang XChen HLi JHu X(2023)Collaborative Graph Neural Networks for Attributed Network EmbeddingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.329800236:3(972-986)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3298002
Zhang XLiu HLi QWu XZhang X(2023)Adaptive Graph Convolution Methods for Attributed Graph ClusteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.327872135:12(12384-12399)Online publication date: 22-May-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3278721
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents