research-article

Compact Graph Structure Learning via Mutual Information Compression

Authors:

Chuan ShiAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1601 - 1610

https://doi.org/10.1145/3485447.3512206

Published: 25 April 2022 Publication History

Abstract

Graph Structure Learning (GSL) recently has attracted considerable attentions in its capacity of optimizing graph structure as well as learning suitable parameters of Graph Neural Networks (GNNs) simultaneously. Current GSL methods mainly learn an optimal graph structure (final view) from single or multiple information sources (basic views), however the theoretical guidance on what is the optimal graph structure is still unexplored. In essence, an optimal graph structure should only contain the information about tasks while compress redundant noise as much as possible, which is defined as ”minimal sufficient structure”, so as to maintain the accurancy and robustness. How to obtain such structure in a principled way? In this paper, we theoretically prove that if we optimize basic views and final view based on mutual information, and keep their performance on labels simultaneously, the final view will be a minimal sufficient structure. With this guidance, we propose a Compact GSL architecture by MI compression, named CoGSL. Specifically, two basic views are extracted from original graph as two inputs of the model, which are refinedly reestimated by a view estimator. Then, we propose an adaptive technique to fuse estimated views into the final view. Furthermore, we maintain the performance of estimated views and the final view and reduce the mutual information of every two views. To comprehensively evaluate the performance of CoGSL, we conduct extensive experiments on several datasets under clean and attacked conditions, which demonstrate the effectiveness and robustness of CoGSL.

References

[1]

Mohammed AlQuraishi. 2019. AlphaFold at CASP13. Bioinformatics (2019), 4862–4865.

[2]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2014. Spectral Networks and Locally Connected Networks on Graphs. In 2nd International Conference on Learning Representations.

[3]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML. 1597–1607.

[4]

Yu Chen, Lingfei Wu, and Mohammed Zaki. 2020. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. NeurIPS (2020).

[5]

Yu Chen, Lingfei Wu, and Mohammed J Zaki. 2019. Reinforcement learning based graph-to-sequence model for natural question generation. arXiv preprint arXiv:1908.04942(2019).

[6]

Thomas M Cover, Joy A Thomas, 1991. Entropy, relative entropy and mutual information. Elements of information theory(1991), 12–13.

[7]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016. 3837–3845.

[8]

Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428(2019).

[9]

Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning discrete structures for graph neural networks. In ICML. 1972–1982.

[10]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. 1263–1272.

[11]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS. 249–256.

[12]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1025–1035.

[13]

Mark S Handcock and Krista J Gile. 2010. Modeling social networks from sampled data. The Annals of Applied Statistics(2010), 5.

[14]

Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In ICML. 4116–4126.

[15]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729–9738.

[16]

Bo Jiang, Ziyan Zhang, Doudou Lin, Jin Tang, and Bin Luo. 2019. Semi-supervised learning with graph learning-convolutional networks. In CVPR. 11313–11320.

[17]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph structure learning for robust graph neural networks. In SIGKDD. 66–74.

[18]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.

[19]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.

[20]

Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR.

[21]

Hu Linmei, Tianchi Yang, Chuan Shi, Houye Ji, and Xiaoli Li. 2019. Heterogeneous graph attention networks for semi-supervised short text classification. In EMNLP-IJCNLP. 4821–4830.

[22]

Dean Lusher, Johan Koskinen, and Garry Robins. 2013. Exponential random graph models for social networks: Theory, methods, and applications. Vol. 35. Cambridge University Press.

[23]

Peter V Marsden. 1990. Network data and measurement. Annual review of sociology(1990), 435–463.

[24]

Péter Mernyei and Cătălina Cangea. 2020. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901(2020).

[25]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748(2018).

[26]

Liam Paninski. 2003. Estimation of entropy and mutual information. Neural computation (2003), 1191–1253.

[27]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research(2011), 2825–2830.

[28]

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-GCN: Geometric Graph Convolutional Networks. In ICLR.

[29]

Xiaojuan Qi, Renjie Liao, Jiaya Jia, Sanja Fidler, and Raquel Urtasun. 2017. 3d graph neural networks for rgbd semantic segmentation. In ICCV. 5199–5208.

[30]

Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. 2017. A simple neural network module for relational reasoning. arXiv preprint arXiv:1706.01427(2017).

[31]

Stefano Soatto and Alessandro Chiuso. 2016. Modeling Visual Representations: Defining Properties and Deep Approximations. In ICLR.

[32]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903(2017).

[33]

Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018. Deep graph infomax. arXiv preprint arXiv:1809.10341(2018).

[34]

Ruijia Wang, Shuai Mou, Xiao Wang, Wanpeng Xiao, Qi Ju, Chuan Shi, and Xing Xie. 2021. Graph Structure Estimation Neural Networks. In WWW. 342–353.

[35]

Tailin Wu, Hongyu Ren, Pan Li, and Jure Leskovec. 2020. Graph information bottleneck. arXiv preprint arXiv:2010.12811(2020).

[36]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2021. A Comprehensive Survey on Graph Neural Networks. IEEE Trans. Neural Networks Learn. Syst.(2021), 4–24.

[37]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In ICLR.

[38]

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation Learning on Graphs with Jumping Knowledge Networks. In ICML. 5449–5458.

[39]

Jiaxuan You, Rex Ying, and Jure Leskovec. 2019. Position-aware graph neural networks. In ICML. 7134–7143.

[40]

Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. NeurIPS (2018), 5165–5175.

[41]

Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. 2021. Heterogeneous Graph Structure Learning for Graph Neural Networks. In AAAI.

[42]

Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. 2020. Robust graph representation learning via neural sparsification. In ICML. 11458–11468.

[43]

Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. 2021. Deep Graph Structure Learning for Robust Representations: A Survey. arXiv preprint arXiv:2103.03036(2021).

[44]

Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In WWW. 2069–2080.

Cited By

In YYoon KKim KShin KPark CChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Self-Guided Robust Graph Structure RefinementProceedings of the ACM on Web Conference 202410.1145/3589334.3645522(697-708)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645522
Zhang PZhang YPiao XSun YYin B(2024)Relationship updating network with contrastive learningPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2024.129874646(129874)Online publication date: Jul-2024
https://doi.org/10.1016/j.physa.2024.129874
Wang ZFan H(2024)Multi-view graph pooling with coarsened graph disentanglementNeural Networks10.1016/j.neunet.2024.106221174:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106221
Show More Cited By

Index Terms

Compact Graph Structure Learning via Mutual Information Compression

Index terms have been assigned to the content through auto-classification.

Recommendations

Towards Unsupervised Deep Graph Structure Learning
WWW '22: Proceedings of the ACM Web Conference 2022

In recent years, graph neural networks (GNNs) have emerged as a successful tool in a variety of graph-related applications. However, the performance of GNNs can be deteriorated when noisy connections occur in the original graph structures; besides, the ...
Graph Structure Estimation Neural Networks
WWW '21: Proceedings of the Web Conference 2021

Graph Neural Networks (GNNs) have drawn considerable attention in recent years and achieved outstanding performance in many tasks. Most empirical studies of GNNs assume that the observed graph represents a complete and accurate picture of node ...
Long-tailed graph neural networks via graph structure learning for node classification
Abstract
Long-tailed methods have gained increasing attention and achieved excellent performance due to the long-tailed distribution in graphs, i.e., many small-degree tail nodes have limited structural connectivity. However, real-world graphs are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

the National Natural Science Foundation of China
The Fundamental Research Funds for the Central Universities

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
577
Total Downloads

Downloads (Last 12 months)185
Downloads (Last 6 weeks)22

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

In YYoon KKim KShin KPark CChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Self-Guided Robust Graph Structure RefinementProceedings of the ACM on Web Conference 202410.1145/3589334.3645522(697-708)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645522
Zhang PZhang YPiao XSun YYin B(2024)Relationship updating network with contrastive learningPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2024.129874646(129874)Online publication date: Jul-2024
https://doi.org/10.1016/j.physa.2024.129874
Wang ZFan H(2024)Multi-view graph pooling with coarsened graph disentanglementNeural Networks10.1016/j.neunet.2024.106221174:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106221
Pan ZCai FChen WShao TGuo YChen H(2024)Inductive link prediction on temporal networks through causal inferenceInformation Sciences10.1016/j.ins.2024.121202(121202)Online publication date: Jul-2024
https://doi.org/10.1016/j.ins.2024.121202
Feng ZMao KZhou H(2024)Adaptive micro- and macro-knowledge incorporation for hierarchical text classificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123374248:COnline publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123374
Li ZWang LSun XLuo YZhu YChen DLuo YZhou XLiu QWu SWang LYu JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)GSLBProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667441(30306-30318)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667441
Zhou ZZhou SMao BZhou XChen JTan QZha DFeng YChen CWang COh ANaumann TGloberson ASaenko KHardt MLevine S(2023)OpenGSLProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666909(17904-17928)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666909
Zhang YWu LShen QPang YWei ZXu FChang ELong B(2023)Graph Learning Augmented Heterogeneous Graph Neural Network for Social RecommendationACM Transactions on Recommender Systems10.1145/36104071:4(1-22)Online publication date: 3-Oct-2023
https://dl.acm.org/doi/10.1145/3610407
Chen YQiang SHa MLiu XLi STong JYuan LGuo XZhu Z(2023)Semi-Supervised Heterogeneous Graph Learning with Multi-Level Data AugmentationACM Transactions on Knowledge Discovery from Data10.1145/360895318:2(1-27)Online publication date: 14-Nov-2023
https://dl.acm.org/doi/10.1145/3608953
Wang HFu YYu THu LJiang WPu SSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)PROSE: Graph Structure Learning via Progressive StrategyProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599476(2337-2348)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599476
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents