research-article

Public Access

Exploring Edge Disentanglement for Node Classification

Authors:

Tianxiang Zhao,

Suhang WangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1028 - 1036

https://doi.org/10.1145/3485447.3511929

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Edges in real-world graphs are typically formed by a variety of factors and carry diverse relation semantics. For example, connections in a social network could indicate friendship, being colleagues, or living in the same neighborhood. However, these latent factors are usually concealed behind mere edge existence due to the data collection and graph formation processes. Despite rapid developments in graph learning over these years, most models take a holistic approach and treat all edges as equal. One major difficulty in disentangling edges is the lack of explicit supervisions. In this work, with close examination of edge patterns, we propose three heuristics and design three corresponding pretext tasks to guide the automatic edge disentanglement. Concretely, these self-supervision tasks are enforced on a designed edge disentanglement module to be trained jointly with the downstream node classification task to encourage automatic edge disentanglement. Channels of the disentanglement module are expected to capture distinguishable relations and neighborhood interactions, and outputs from them are aggregated as node representations. The proposed is easy to be incorporated with various neural architectures, and we conduct experiments on 6 real-world datasets. Empirical results show that it can achieve significant performance gains.

References

[1]

Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learning. PMLR, 21–29.

[2]

James Atwood and Don Towsley. 2016. Diffusion-convolutional neural networks. In Advances in neural information processing systems. 1993–2001.

[3]

Shaked Brody, Uri Alon, and Eran Yahav. 2021. How Attentive are Graph Attention Networks?arXiv preprint arXiv:2105.14491(2021).

[4]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203(2013).

[5]

Hryhorii Chereda, A. Bleckmann, F. Kramer, A. Leha, and T. Beißbarth. 2019. Utilizing Molecular Network Information via Graph Convolutional Neural Networks to Predict Metastatic Event in Breast Cancer. Studies in health technology and informatics 267 (2019), 181–186.

[6]

Enyan Dai, Charu Aggarwal, and Suhang Wang. 2021. NRGNN: Learning a Label Noise Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphs. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 227–236.

Digital Library

[7]

Enyan Dai, Wei Jin, Hui Liu, and Suhang Wang. 2022. Towards Robust Graph Neural Networks for Noisy Graphs with Sparse Labels. arXiv preprint arXiv:2201.00232(2022).

[8]

David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems. 2224–2232.

[9]

Wenqi Fan, Y. Ma, Qing Li, Yuan He, Y. Zhao, Jiliang Tang, and D. Yin. 2019. Graph Neural Networks for Social Recommendation. The World Wide Web Conference(2019).

[10]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural Message Passing for Quantum Chemistry. In ICML.

[11]

William L. Hamilton, Zhitao Ying, and J. Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS.

[12]

Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2019. Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations.

[13]

Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, and Jiliang Tang. 2020. Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006.10141(2020).

[14]

Longlong Jing and Yingli Tian. 2020. Self-supervised visual feature learning with deep neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence (2020).

[15]

Justin M Johnson and Taghi M Khoshgoftaar. 2019. Survey on deep learning with class imbalance. Journal of Big Data 6, 1 (2019), 27.

[16]

Dongkwan Kim and Alice Oh. 2020. How to find your friendly neighborhood: Graph attention design with self-supervision. In International Conference on Learning Representations.

[17]

Thomas Kipf and M. Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. ArXiv abs/1609.02907(2017).

[18]

Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942(2019).

[19]

Yixin Liu, Shirui Pan, Ming Jin, Chuan Zhou, Feng Xia, and Philip S Yu. 2021. Graph self-supervised learning: A survey. arXiv preprint arXiv:2103.00111(2021).

[20]

Yanbei Liu, Xiao Wang, Shu Wu, and Zhitao Xiao. 2020. Independence promoted graph disentangled networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 4916–4923.

[21]

Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. 2019. Disentangled graph convolutional networks. In International Conference on Machine Learning. PMLR, 4212–4221.

[22]

Elman Mansimov, O. Mahmood, Seokho Kang, and Kyunghyun Cho. 2019. Molecular Geometry Prediction using a Deep Generative Graph Neural Network. Scientific Reports 9(2019).

[23]

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2019. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations.

[24]

Bryan Perozzi, Rami Al-Rfou, and S. Skiena. 2014. DeepWalk: online learning of social representations. In KDD ’14.

[25]

Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1150–1160.

Digital Library

[26]

Neelam Rout, Debahuti Mishra, and Manas Kumar Mallick. 2018. Handling imbalanced data: A survey. In International Proceedings on Advances in Soft Computing, Intelligent Systems and Applications. Springer, 431–443.

[27]

Daniil Sorokin and Iryna Gurevych. 2018. Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering. ArXiv abs/1808.04126(2018).

[28]

S. Tang, Bo Li, and Haijun Yu. 2019. ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units using Chebyshev Approximations. ArXiv abs/1911.05467(2019).

[29]

Petar Velickovic, Guillem Cucurull, A. Casanova, A. Romero, P. Liò, and Yoshua Bengio. 2018. Graph Attention Networks. ArXiv abs/1710.10903(2018).

[30]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heterogeneous graph attention network. In The World Wide Web Conference. 2022–2032.

Digital Library

[31]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems (2020).

[32]

Teng Xiao, Zhengyu Chen, Donglin Wang, and Suhang Wang. 2021. Learning how to propagate messages in graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1894–1903.

Digital Library

[33]

Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji. 2021. Self-supervised learning of graph neural networks: A unified review. arXiv preprint arXiv:2102.10757(2021).

[34]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.

[35]

Yiding Yang, Zunlei Feng, Mingli Song, and Xinchao Wang. 2020. Factorizable Graph Convolutional Networks. Advances in Neural Information Processing Systems 33 (2020).

[36]

Jiaqi Zeng and Pengtao Xie. 2020. Contrastive self-supervised learning for graph classification. arXiv (2020).

[37]

Tianxiang Zhao, Xianfeng Tang, Xiang Zhang, and Suhang Wang. 2020. Semi-Supervised Graph-to-Graph Translation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1863–1872.

Digital Library

[38]

Tianxiang Zhao, Xiang Zhang, and Suhang Wang. 2021. GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks. In Proceedings of the Fourteenth ACM International Conference on Web Search and Data Mining.

Digital Library

[39]

T. Zhong, Tianliang Wang, Jiahao Wang, J. Wu, and Fan Zhou. 2020. Multiple-Aspect Attentional Graph Neural Networks for Online Social Network User Localization. IEEE Access 8(2020), 95223–95234.

[40]

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. 2020. Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs. Advances in Neural Information Processing Systems 33 (2020).

Cited By

Zhao TZhang XWang SChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Disambiguated Node Classification with Graph Neural NetworksProceedings of the ACM on Web Conference 202410.1145/3589334.3645637(914-923)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645637
Zhu YLi JZheng ZChen LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Fair Graph Representation Learning via Sensitive Attribute DisentanglementProceedings of the ACM on Web Conference 202410.1145/3589334.3645532(1182-1192)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645532
Zheng SZhu ZLiu ZCheng JZhao Y(2024)Adversarial Graph Disentanglement With Component-Specific AggregationIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33162025:5(2204-2216)Online publication date: May-2024
https://doi.org/10.1109/TAI.2023.3316202
Show More Cited By

Index Terms

Exploring Edge Disentanglement for Node Classification
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
2. Information systems
  1. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Despite the success of Graph Neural Networks (GNNs) on various applications, GNNs encounter significant performance degradation when the amount of supervision signals, i.e., number of labeled nodes, is limited, which is expected as GNNs are trained ...
Label-Consistency based Graph Neural Networks for Semi-supervised Node Classification
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Graph neural networks (GNNs) achieve remarkable success in graph-based semi-supervised node classification, leveraging the information from neighboring nodes to improve the representation learning of target node. The success of GNNs at node ...
CLNode: Curriculum Learning for Node Classification
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining

Node classification is a fundamental graph-based task that aims to predict the classes of unlabeled nodes, for which Graph Neural Networks (GNNs) are the state-of-the-art methods. Current GNNs assume that nodes in the training set contribute equally ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
745
Total Downloads

Downloads (Last 12 months)306
Downloads (Last 6 weeks)14

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao TZhang XWang SChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Disambiguated Node Classification with Graph Neural NetworksProceedings of the ACM on Web Conference 202410.1145/3589334.3645637(914-923)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645637
Zhu YLi JZheng ZChen LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Fair Graph Representation Learning via Sensitive Attribute DisentanglementProceedings of the ACM on Web Conference 202410.1145/3589334.3645532(1182-1192)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645532
Zheng SZhu ZLiu ZCheng JZhao Y(2024)Adversarial Graph Disentanglement With Component-Specific AggregationIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33162025:5(2204-2216)Online publication date: May-2024
https://doi.org/10.1109/TAI.2023.3316202
Zou YFang ZWu ZZheng CWang S(2024)Revisiting multi-view learningNeural Networks10.1016/j.neunet.2023.10.052169:C(496-505)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.neunet.2023.10.052
Ko GJung J(2024)Learning disentangled representations in signed directed graphs without social assumptionsInformation Sciences: an International Journal10.1016/j.ins.2024.120373665:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120373
Guo QYang XZhang FXu T(2024)Perturbation-augmented Graph Convolutional NetworksEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107616129:COnline publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107616
Zhao TLuo DZhang XWang S(2023)Faithful and Consistent Graph Neural Network Explanations with Rationale AlignmentACM Transactions on Intelligent Systems and Technology10.1145/361654214:5(1-23)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3616542
Zhang KCao QFang GXu BZou HShen HCheng XSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)DyTed: Disentangled Representation Learning for Discrete-time Dynamic GraphProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599319(3309-3320)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599319
Zhang XFu JLi S(2023)Contrastive Disentangled Learning on Graph for Node Classification2023 IEEE 13th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER)10.1109/CYBER59472.2023.10256657(23-28)Online publication date: 11-Jul-2023
https://doi.org/10.1109/CYBER59472.2023.10256657
Ren WWang LLiu KGuo RPeng LFu Y(2022)Mitigating Popularity Bias in Recommendation with Unbalanced Interactions: A Gradient Perspective2022 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM54844.2022.00054(438-447)Online publication date: Nov-2022
https://doi.org/10.1109/ICDM54844.2022.00054

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents