research-article

GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily

Authors:

Dongmei ZhangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1550 - 1558

https://doi.org/10.1145/3485447.3512201

Published: 25 April 2022 Publication History

Abstract

Graph Neural Networks (GNNs) are widely used on a variety of graph-based machine learning tasks. For node-level tasks, GNNs have strong power to model the homophily property of graphs (i.e., connected nodes are more similar), while their ability to capture heterophily property is often doubtful. This is partially caused by the design of the feature transformation with the same kernel for the nodes in the same hop and the followed aggregation operator. One kernel cannot model the similarity and the dissimilarity (i.e., the positive and negative correlation) between node features simultaneously even though we use attention mechanisms like Graph Attention Network (GAT), since the weight calculated by attention is always a positive value. In this paper, we propose a novel GNN model based on a bi-kernel feature transformation and a selection gate. Two kernels capture homophily and heterophily information respectively, and the gate is introduced to select which kernel we should use for the given node pairs. We conduct extensive experiments on various datasets with different homophily-heterophily properties. The experimental results show consistent and significant improvements against state-of-the-art GNN methods.

References

[1]

Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019).

[2]

Smriti Bhagat, Graham Cormode, and S Muthukrishnan. 2011. Node Classification in Social Networks. arXiv: Social and Information Networks(2011), 115–148.

[3]

Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020. Simple and Deep Graph Convolutional Networks. CoRR abs/2007.02133(2020). arXiv:2007.02133https://arxiv.org/abs/2007.02133

[4]

Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020. Simple and Deep Graph Convolutional Networks. In ICML(Proceedings of Machine Learning Research, Vol. 119). PMLR, 1725–1735.

[5]

David L Davies and Donald W Bouldin. 1979. A cluster separation measure. IEEE transactions on pattern analysis and machine intelligence2 (1979), 224–227.

Digital Library

[6]

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016), 3844–3852.

[7]

Lun Du, Fei Gao, Xu Chen, Ran Jia, Junshan Wang, Jiang Zhang, Shi Han, and Dongmei Zhang. 2021. TabularNet: A Neural Network Architecture for Understanding Semantic Structures of Tabular Data. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 322–331.

Digital Library

[8]

Gintare Karolina Dziugaite and Daniel M Roy. 2017. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv:1703.11008(2017).

[9]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. CoRR abs/1706.02216(2017). arXiv:1706.02216http://arxiv.org/abs/1706.02216

[10]

William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017).

Digital Library

[11]

Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2016. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836(2016).

[12]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016).

[13]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR 2017).

[14]

Qingqing Long, Yiming Wang, Lun Du, Guojie Song, Yilun Jin, and Wei Lin. 2019. Hierarchical community structure preserving network embedding: A subspace approach. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 409–418.

Digital Library

[15]

Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In ICLR.

[16]

Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. 2021. Is Homophily a Necessity for Graph Neural Networks?arXiv preprint arXiv:2106.06134(2021).

[17]

Sunil Kumar Maurya, Xin Liu, and Tsuyoshi Murata. 2021. Improving Graph Neural Networks with Simple Architecture Design. arXiv preprint arXiv:2105.07634(2021).

[18]

Parth Natekar and Manik Sharma. 2020. Representation based complexity measures for predicting generalization in deep learning. arXiv preprint arXiv:2012.02775(2020).

[19]

Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. 2017. Exploring generalization in deep learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 5949–5958.

[20]

Behnam Neyshabur, Ruslan Salakhutdinov, and Nathan Srebro. 2015. Path-SGD: path-normalized optimization in deep neural networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2. 2422–2430.

[21]

Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. 2015. Norm-based capacity control in neural networks. In Conference on Learning Theory. PMLR, 1376–1401.

[22]

Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2019. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations.

[23]

Wenbin Tang, Honglei Zhuang, and Jie Tang. 2011. Learning to infer social ties in large networks. In Joint european conference on machine learning and knowledge discovery in databases. Springer, 381–397.

[24]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903(2017).

[25]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR 2018).

[26]

Junshan Wang, Zhicong Lu, Guojia Song, Yue Fan, Lun Du, and Wei Lin. 2019. Tag2vec: Learning tag representations in tag networks. In The World Wide Web Conference. 3314–3320.

Digital Library

[27]

Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. Am-gcn: Adaptive multi-channel graph convolutional networks. In Proceedings of the 26th ACM SIGKDD International conference on knowledge discovery & data mining. 1243–1253.

Digital Library

[28]

Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. AM-GCN: Adaptive Multi-channel Graph Convolutional Networks. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (Eds.). ACM, 1243–1253. https://doi.org/10.1145/3394486.3403177

Digital Library

[29]

Yanlin Wang, Lun Du, Ensheng Shi, Yuxuan Hu, Shi Han, and Dongmei Zhang. 2020. Cocogum: Contextual code summarization with multi-relational gnn on umls. Microsoft, MSR-TR-2020-16. [Online]. Available: https://www.microsoft.com/en-us/research/publication/cocogum-contextual-code-summarization-with-multi-relational-gnn-on-umls(2020).

[30]

Yun Wang, Lun Du, Guojie Song, Xiaojun Ma, Lichen Jin, Wei Lin, and Fei Sun. 2019. Tag2Gauss: Learning Tag Representations via Gaussian Distribution in Tagged Networks. In IJCAI. 3799–3805.

[31]

Lanning Wei, Huan Zhao, and Zhiqiang He. 2021. Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspective. arXiv preprint arXiv:2112.14531(2021).

[32]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.

[33]

Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. 2021. Self-supervised Graph-level Representation Learning with Local and Global Structure. arXiv preprint arXiv:2106.04113(2021).

[34]

Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. 2016. Revisiting Semi-Supervised Learning with Graph Embeddings. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016(JMLR Workshop and Conference Proceedings, Vol. 48), Maria-Florina Balcan and Kilian Q. Weinberger (Eds.). JMLR.org, 40–48. http://proceedings.mlr.press/v48/yanga16.html

[35]

Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. 2016. Revisiting Semi-Supervised Learning with Graph Embeddings. CoRR abs/1603.08861(2016). arXiv:1603.08861http://arxiv.org/abs/1603.08861

[36]

Jiong Zhu, Ryan A Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K Ahmed, and Danai Koutra. 2020. Graph neural networks with heterophily. arXiv preprint arXiv:2009.13566(2020).

[37]

Jiong Zhu, Ryan A. Rossi, Anup B. Rao, Tung Mai, Nedim Lipka, Nesreen K. Ahmed, and Danai Koutra. 2020. Graph Neural Networks with Heterophily. CoRR abs/2009.13566(2020). arXiv:2009.13566https://arxiv.org/abs/2009.13566

[38]

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. 2020. Beyond homophily in graph neural networks: Current limitations and effective designs. arXiv preprint arXiv:2006.11468(2020).

Cited By

Khoshraftar SAn A(2024)A Survey on Graph Representation Learning MethodsACM Transactions on Intelligent Systems and Technology10.1145/363351815:1(1-55)Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1145/3633518
Zhang WYan GShen YLing YTao YCui BTang JBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)NPA: Improving Large-scale Graph Neural Networks with Non-parametric AttentionCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3653399(414-427)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3653399
Cheng MXu HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)A Quasi-Wasserstein Loss for Learning Graph Neural NetworksProceedings of the ACM on Web Conference 202410.1145/3589334.3645586(815-826)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645586
Show More Cited By

Index Terms

GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily

Index terms have been assigned to the content through auto-classification.

Recommendations

H2-FDetector: A GNN-based Fraud Detector with Homophilic and Heterophilic Connections
WWW '22: Proceedings of the ACM Web Conference 2022

In the fraud graph, fraudsters often interact with a large number of benign entities to hide themselves. So, there are not only the homophilic connections formed by the same label nodes (similar nodes), but also the heterophilic connections formed by ...
Investigating Homophily in Online Social Networks
WI-IAT '10: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01

Similarity breeds connections, the principle of homophily, has been well studied in existing sociology literature. %Several studies have observed this phenomena by conducting surveys on human subjects. These studies have concluded that new ties are ...
How does Heterophily Impact the Robustness of Graph Neural Networks?: Theoretical Connections and Practical Implications
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

We bridge two research directions on graph neural networks (GNNs), by formalizing the relation between heterophily of node labels (i.e., connected nodes tend to have dissimilar labels) and the robustness of GNNs to adversarial attacks. Our theoretical ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
611
Total Downloads

Downloads (Last 12 months)217
Downloads (Last 6 weeks)9

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Khoshraftar SAn A(2024)A Survey on Graph Representation Learning MethodsACM Transactions on Intelligent Systems and Technology10.1145/363351815:1(1-55)Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1145/3633518
Zhang WYan GShen YLing YTao YCui BTang JBarcelo PSanchez-Pi NMeliou ASudarshan S(2024)NPA: Improving Large-scale Graph Neural Networks with Non-parametric AttentionCompanion of the 2024 International Conference on Management of Data10.1145/3626246.3653399(414-427)Online publication date: 9-Jun-2024
https://dl.acm.org/doi/10.1145/3626246.3653399
Cheng MXu HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)A Quasi-Wasserstein Loss for Learning Graph Neural NetworksProceedings of the ACM on Web Conference 202410.1145/3589334.3645586(815-826)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645586
Li XWu ZZhang WSun HLi RWang G(2024)AdaFGL: A New Paradigm for Federated Node Classification with Topology Heterogeneity2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00198(2517-2530)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00198
Peng TWu WYuan HBao ZPengru ZYu XLin XLiang YPu Y(2024)GraphRARE: Reinforcement Learning Enhanced Graph Neural Network with Relative Entropy2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00196(2489-2502)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00196
Sun HLi XWu ZSu DLi RWang G(2024)Breaking the Entanglement of Homophily and Heterophily in Semi-supervised Node Classification2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00188(2379-2392)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00188
Munir SMalick RFerretti S(2024)A Network Analysis-Driven Framework for Factual Explainability of Knowledge GraphsIEEE Access10.1109/ACCESS.2024.336797112(28071-28082)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3367971
Ai GYan HWang HLi X(2024)A2GCN: Graph Convolutional Networks with Adaptive Frequency and Arbitrary OrderPattern Recognition10.1016/j.patcog.2024.110764156(110764)Online publication date: Dec-2024
https://doi.org/10.1016/j.patcog.2024.110764
Wang JGuo YYang LWang Y(2024)Heterophily-aware graph attention networkPattern Recognition10.1016/j.patcog.2024.110738156(110738)Online publication date: Dec-2024
https://doi.org/10.1016/j.patcog.2024.110738
Wu JHu RLi DRen LHuang ZZang Y(2024)Beyond the individualNeural Networks10.1016/j.neunet.2023.10.019169:C(20-31)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.neunet.2023.10.019
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents