research-article

Unsupervised Hierarchical Graph Pooling via Substructure-Sensitive Mutual Information Maximization

Authors:

Hongzuo XuAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 1299 - 1308

https://doi.org/10.1145/3511808.3557485

Published: 17 October 2022 Publication History

Abstract

Graph pooling plays a vital role in learning graph embeddings. Due to the lack of label information, unsupervised graph pooling has received much attention, primarily via mutual information (MI). However, most existing MI-based pooling methods only preserve node features while overlooking the hierarchical substructural information. In this paper, we propose SMIP, a novel unsupervised hierarchical graph pooling method based on substructure-sensitive MI maximization. SMIP reconstructs a hard-style substructure encoder based on cluster-based pooling paradigm, and trains it with two substructure-sensitive MI-based objectives, i.e., node-substructure MI and node-node MI. The node-substructure MI guides to transfer maximum node feature information into corresponded substructures and the node-node MI guarantees a more accurate node allocation. Moreover, to avoid extra computation of augmented graphs and prevent noise information during MI estimation, we propose a local-scope contrastive MI estimation method, making SMIP more potent in capturing intrinsic features of the input graph. Experiments on six benchmark graph classification datasets demonstrate that our hierarchical deep learning approach outperforms all state-of-the-art unsupervised GNN-based methods and even surpasses the performance of nine supervised ones. Generalization study shows that the proposed substructure-sensitive MI objective can be successfully embedded into other cluster-based pooling methods to improve their performance.

Supplementary Material

MP4 File (CIKM22-fp0805.mp4)

Presentation video of CIKM 2022 paper "Unsupervised Hierarchical Graph Pooling via Substructure-Sensitive Mutual Information Maximization"

Download
31.70 MB

References

[1]

Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, and B Aditya Prakash. 2018. Sub2vec: Feature learning for subgraphs. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 170--182.

[2]

Yunsheng Bai, Hao Ding, Yang Qiao, Agustin Marinovic, Ken Gu, Ting Chen, Yizhou Sun, and Wei Wang. 2019. Unsupervised inductive graph-level representation learning via graph-graph proximity. arXiv preprint arXiv:1904.01098 (2019).

[3]

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In ICML. PMLR, 531--540.

[4]

Filippo Maria Bianchi, Daniele Grattarola, and Cesare Alippi. 2020. Spectral clustering with graph neural networks for graph pooling. In ICML. PMLR, 874--883.

[5]

Karsten M Borgwardt and Hans-Peter Kriegel. 2005. Shortest-path kernels on graphs. In Fifth IEEE international conference on data mining (ICDM'05). IEEE, 8--pp.

Digital Library

[6]

Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. 2005. Protein function prediction via graph kernels. Bioinformatics, Vol. 21, suppl_1 (2005), i47--i56.

Digital Library

[7]

Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. 2020. Club: A contrastive log-ratio upper bound of mutual information. In International conference on machine learning. PMLR, 1779--1788.

[8]

Guanyi Chu, Xiao Wang, Chuan Shi, and Xunqiang Jiang. 2021. CuCo: Graph Representation with Curriculum Contrastive Learning. In IJCAI. 2300--2306.

[9]

Thomas M Cover. 1999. Elements of information theory. John Wiley & Sons.

[10]

Fei Ding, Xiaohong Zhang, Justin Sybrandt, and Ilya Safro. 2020. Unsupervised Hierarchical Graph Representation Learning by Mutual Information Maximization. arXiv preprint arXiv:2003.08420 (2020).

[11]

Roger C Entringer, Douglas E Jackson, and DA Snyder. 1976. Distance in graphs. Czechoslovak Mathematical Journal, Vol. 26, 2 (1976), 283--296.

[12]

Pedro F Felzenszwalb and Daniel P Huttenlocher. 2004. Efficient graph-based image segmentation. International journal of computer vision, Vol. 59, 2 (2004), 167--181.

Digital Library

[13]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855--864.

Digital Library

[14]

Michael Gutmann and Aapo Hyv"arinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 297--304.

[15]

Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In ICML. PMLR, 4116--4126.

[16]

R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2018. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018).

[17]

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, Vol. 33 (2020), 22118--22133.

[18]

Jingjia Huang, Zhangheng Li, Nannan Li, Shan Liu, and Ge Li. 2019. Attpool: Towards hierarchical feature representation in graph convolutional networks via attention mechanism. In ICCV. 6480--6489.

[19]

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).

[20]

Thomas Kipf and Max Welling. 2016a. Semi-Supervised Classification with Graph Convolutional Networks. arXiv: Learning (2016).

[21]

Thomas N Kipf and Max Welling. 2016b. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[22]

Risi Kondor and Horace Pan. 2016. The multiscale laplacian graph kernel. arXiv preprint arXiv:1603.06186 (2016).

[23]

Junhyun Lee, Inyeop Lee, and Jaewoo Kang. 2019. Self-attention graph pooling. arXiv preprint arXiv:1904.08082 (2019).

[24]

Maosen Li, Siheng Chen, Ya Zhang, and Ivor W Tsang. 2020. Graph Cross Networks with Vertex Infomax Pooling. arXiv preprint arXiv:2010.01804 (2020).

[25]

Ralph Linsker. 1988. Self-organization in a perceptual network. Computer, Vol. 21, 3 (1988), 105--117.

Digital Library

[26]

Ning Liu, Songlei Jian, Dongsheng Li, Yiming Zhang, Zhiquan Lai, and Hongzuo Xu. 2021. Hierarchical adaptive pooling by capturing high-order dependency for graph representation learning. IEEE Transactions on Knowledge and Data Engineering (2021).

Digital Library

[27]

Tengfei Ma and Jie Chen. 2019. Unsupervised learning of graph hierarchical abstractions with differentiable coarsening and optimal transport. arXiv preprint arXiv:1912.11176 (2019).

[28]

Annamalai Narayanan, Mahinthan Chandramohan, Rajasekar Venkatesan, Lihui Chen, Yang Liu, and Shantanu Jaiswal. 2017. graph2vec: Learning distributed representations of graphs. arXiv preprint arXiv:1707.05005 (2017).

[29]

Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-gan: Training generative neural samplers using variational divergence minimization. In NIPS. 271--279.

[30]

Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference 2020. 259--270.

Digital Library

[31]

Natavs a Prvz ulj. 2007. Biological network comparison using graphlet degree distribution. Bioinformatics, Vol. 23, 2 (2007), e177--e183.

Digital Library

[32]

Ekagra Ranjan, Soumya Sanyal, and Partha P Talukdar. 2020. ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations. In AAAI. 5470--5477.

[33]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC. 593--607.

[34]

Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, Vol. 12, 9 (2011).

[35]

Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. 2009. Efficient graphlet kernels for large graph comparison. In Artificial intelligence and statistics. PMLR, 488--495.

[36]

Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2019. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv preprint arXiv:1908.01000 (2019).

[37]

Alain Trémeau and Philippe Colantoni. 2000. Regions adjacency graph applied to color image segmentation. IEEE T IMAGE PROCESS, Vol. 9, 4 (2000), 735--744.

Digital Library

[38]

Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv e-prints (2018), arXiv--1807.

[39]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).

[40]

Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.

[41]

Chenguang Wang and Ziwen Liu. 2021. Learning graph representation by aggregating subgraphs via mutual information maximization. arXiv preprint arXiv:2103.13125 (2021).

[42]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).

[43]

Pinar Yanardag and SVN Vishwanathan. 2015. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1365--1374.

Digital Library

[44]

Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. arXiv preprint arXiv:1806.08804 (2018).

[45]

Hao Yuan and Shuiwang Ji. 2019. StructPool: Structured graph pooling via conditional random fields. In ICLR.

[46]

Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An End-to-End Deep Learning Architecture for Graph Classification. In AAAI, Vol. 18. 4438--4445.

Cited By

Li ZSu HWu YZhang QYuan CGribova VFilaretov VHuang D(2024)Hierarchical Multiview Top-k Pooling With Deep-Q-NetworksIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33342615:6(2985-2996)Online publication date: Jun-2024
https://doi.org/10.1109/TAI.2023.3334261
Wang YHou WSheng NZhao ZLiu JHuang LWang J(2024)Graph pooling in graph neural networks: methods and their applications in omics studiesArtificial Intelligence Review10.1007/s10462-024-10918-957:11Online publication date: 16-Sep-2024
https://doi.org/10.1007/s10462-024-10918-9
Liu CZhan YWu JLi CDu BHu WLiu TTao DElkind E(2023)Graph pooling for graph neural networksProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/752(6712-6722)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/752
Show More Cited By

Index Terms

Unsupervised Hierarchical Graph Pooling via Substructure-Sensitive Mutual Information Maximization
1. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Graph Pooling via Coarsened Graph Infomax
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Graph pooling that summaries the information in a large graph into a compact form is essential in hierarchical graph representation learning. Existing graph pooling methods either suffer from high computational complexity or cannot capture the global ...
Quasi-CliquePool: Hierarchical Graph Pooling for Graph Classification
SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

Graph Neural Networks (GNNs) have revolutionized graph learning through efficiently learned node embeddings and achieved promising results in various graph-related tasks such as node and graph classification. Within GNNs, a pooling operation reduces the ...
Multi-Granularity Contrastive Learning for Graph with Hierarchical Pooling
Artificial Neural Networks and Machine Learning – ICANN 2023
Abstract
Graph contrastive learning is an unsupervised learning method for graph data. It aims to learn useful representations by maximizing the similarity between similar instances and minimizing the similarity between dissimilar instances. Despite the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National University of Defense Technology Foundation
National Natural Science Foundation of China
State Administration of Science Technology and Industry for National Defense Foundation

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
215
Total Downloads

Downloads (Last 12 months)53
Downloads (Last 6 weeks)8

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li ZSu HWu YZhang QYuan CGribova VFilaretov VHuang D(2024)Hierarchical Multiview Top-k Pooling With Deep-Q-NetworksIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33342615:6(2985-2996)Online publication date: Jun-2024
https://doi.org/10.1109/TAI.2023.3334261
Wang YHou WSheng NZhao ZLiu JHuang LWang J(2024)Graph pooling in graph neural networks: methods and their applications in omics studiesArtificial Intelligence Review10.1007/s10462-024-10918-957:11Online publication date: 16-Sep-2024
https://doi.org/10.1007/s10462-024-10918-9
Liu CZhan YWu JLi CDu BHu WLiu TTao DElkind E(2023)Graph pooling for graph neural networksProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/752(6712-6722)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/752
Shi BWang YGuo FShao JShen HCheng XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Graph Domain Adaptation with Network HierarchyProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614928(2249-2258)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614928
Ye HSong YWang BWu LHe SBo XZhang Z(2023)HSGCL-DTA: Hybrid-scale Graph Contrastive Learning based Drug-Target Binding Affinity Prediction2023 IEEE 35th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI59109.2023.00142(947-954)Online publication date: 6-Nov-2023
https://doi.org/10.1109/ICTAI59109.2023.00142
Lu JSong FWu JWu X(2023)An Unsupervised Graph Embedding Method Based on Dynamic Graph Attention Networks and Infomax for Link Prediction2023 8th International Conference on Data Science in Cyberspace (DSC)10.1109/DSC59305.2023.00065(409-415)Online publication date: 18-Aug-2023
https://doi.org/10.1109/DSC59305.2023.00065

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents