research-article

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

Authors:

Jianling SunAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1506 - 1516

https://doi.org/10.1145/3485447.3512197

Published: 25 April 2022 Publication History

Abstract

The prevalence of graph structures attracts a surge of investigation on graph data, enabling several downstream tasks such as multi-graph classification. However, in the multi-graph setting, graphs usually follow a long-tailed distribution in terms of their sizes, i.e., the number of nodes. In particular, a large fraction of tail graphs usually have small sizes. Though recent graph neural networks (GNNs) can learn powerful graph-level representations, they treat the graphs uniformly and marginalize the tail graphs which suffer from the lack of distinguishable structures, resulting in inferior performance on tail graphs. To alleviate this concern, in this paper we propose a novel graph neural network named SOLT-GNN, to close the representational gap between the head and tail graphs from the perspective of knowledge transfer. In particular, SOLT-GNN capitalizes on the co-occurrence substructures exploitation to extract the transferable patterns from head graphs. Furthermore, a novel relevance prediction function is proposed to memorize the pattern relevance derived from head graphs, in order to predict the complements for tail graphs to attain more comprehensive structures for enrichment. We conduct extensive experiments on five benchmark datasets, and demonstrate that our proposed model can outperform the state-of-the-art baselines.

References

[1]

Daniel Beck, Gholamreza Haffari, and Trevor Cohn. 2018. Graph-to-Sequence Learning using Gated Graph Neural Networks. In ACL. 273–283.

[2]

Karsten M Borgwardt, Cheng Soon Ong, Stefan Schönauer, SVN Vishwanathan, Alex J Smola, and Hans-Peter Kriegel. 2005. Protein function prediction via graph kernels. Bioinformatics 21, suppl_1 (2005), i47–i56.

Digital Library

[3]

Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE TKDE 30, 9 (2018), 1616–1637.

[4]

Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: a library for support vector machines. ACM TIST 2, 3 (2011), 1–27.

Digital Library

[5]

Zhihong Chen, Rong Xiao, Chenliang Li, Gangfeng Ye, Haochuan Sun, and Hongbo Deng. 2020. ESAM: Discriminative Domain Adaptation with Non-Displayed Items to Improve Long-Tail Performance. In SIGIR. 579–588.

Digital Library

[6]

Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. 2020. Graph prototypical networks for few-shot learning on attributed networks. In CIKM. 295–304.

[7]

Paul D Dobson and Andrew J Doig. 2003. Distinguishing enzyme structures from non-enzymes without alignments. Journal of Molecular Biology 330, 4 (2003), 771–783.

[8]

David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In NeurIPS. 2224–2232.

[9]

Hongyang Gao and Shuiwang Ji. 2019. Graph u-nets. In ICML. 2083–2092.

[10]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. 1263–1272.

[11]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD. 855–864.

[12]

David Ha, Andrew Dai, and Quoc V Le. 2017. Hypernetworks. In ICLR.

[13]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1025–1035.

[14]

Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. 2020. Decoupling representation and classifier for long-tailed recognition. In ICLR.

[15]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.

[16]

Nils M Kriege, Fredrik D Johansson, and Christopher Morris. 2020. A survey on graph kernels. Applied Network Science 5, 1 (2020), 1–42.

[17]

Junhyun Lee, Inyeop Lee, and Jaewoo Kang. 2019. Self-attention graph pooling. In ICML. 3734–3743.

[18]

Yanyan Liang, Yanfeng Zhang, Dechao Gao, and Qian Xu. 2021. An End-to-End Multiplex Graph Neural Network for Graph Representation Learning. IEEE Access 9(2021), 58861–58869.

[19]

Jialun Liu, Yifan Sun, Chuchu Han, Zhaopeng Dou, and Wenhui Li. 2020. Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective. In CVPR. 2970–2979.

[20]

Siyi Liu and Yujia Zheng. 2020. Long-tail session-based recommendation. In RecSys. 509–514.

[21]

Zemin Liu, Yuan Fang, Chenghao Liu, and Steven C.H. Hoi. 2021. Node-wise Localization of Graph Neural Networks. In IJCAI. 1520–1526.

[22]

Zemin Liu, Yuan Fang, Chenghao Liu, and Steven CH Hoi. 2021. Relative and Absolute Location Embedding for Few-Shot Node Classification on Graph. In AAAI.

[23]

Zemin Liu, Yuan Fang, Yong Liu, and Vincent W. Zheng. 2021. Neighbor-Anchoring Adversarial Graph Neural Networks. IEEE TKDE Early Access(2021).

[24]

Zemin Liu, Trung-Kien Nguyen, and Yuan Fang. 2021. Tail-GNN: Tail-Node Graph Neural Networks. In KDD. 1109–1119.

[25]

Zemin Liu, Wentao Zhang, Yuan Fang, Xinming Zhang, and Steven CH Hoi. 2020. Towards locality-aware meta-learning of tail node embeddings on networks. In CIKM. 975–984.

[26]

Yao Ma, Suhang Wang, Charu C Aggarwal, and Jiliang Tang. 2019. Graph convolutional networks with eigenpooling. In KDD. 723–731.

[27]

Diego Mesquita, Amauri Souza, and Samuel Kaski. 2020. Rethinking pooling in graph neural networks. In NeurIPS.

[28]

Francesco Orsini, Paolo Frasconi, and Luc De Raedt. 2015. Graph invariant kernels. In IJCAI. 3756–3762.

[29]

Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. 2018. FiLM: Visual reasoning with a general conditioning layer. In AAAI. 3942–3951.

[30]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In KDD. 701–710.

[31]

Liang Qu, Huaisheng Zhu, Ruiqi Zheng, Yuhui Shi, and Hongzhi Yin. 2021. ImGAGN: Imbalanced Network Embedding via Generative Adversarial Graph Networks. In KDD. 1390–1398.

[32]

Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar. 2020. Asap: Adaptive structure aware pooling for learning hierarchical graph representations. In AAAI. 5470–5477.

[33]

Robert Sanders. 1987. The Pareto principle: its use and abuse. Journal of Services Marketing 1, 2 (1987), 37–40.

[34]

Aravind Sankar, Junting Wang, Adit Krishnan, and Hari Sundaram. 2021. ProtoCF: Prototypical Collaborative Filtering for Few-shot Recommendation. In RecSys. 166–175.

[35]

Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-Lehman graph kernels.Journal of Machine Learning Research 12, 9 (2011).

[36]

Nino Shervashidze, SVN Vishwanathan, Tobias Petri, Kurt Mehlhorn, and Karsten Borgwardt. 2009. Efficient graphlet kernels for large graph comparison. In AISTATS. 488–495.

[37]

Min Shi, Yufei Tang, Xingquan Zhu, David Wilson, and Jianxun Liu. 2020. Multi-class imbalanced graph convolutional network learning. In IJCAI.

[38]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In WWW. 1067–1077.

Digital Library

[39]

Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. 2020. Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect. In NeurIPS.

[40]

Hannu Toivonen, Ashwin Srinivasan, Ross D King, Stefan Kramer, and Christoph Helma. 2003. Statistical evaluation of the predictive toxicology challenge 2000–2001. Bioinformatics 19, 10 (2003), 1183–1193.

[41]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR.

[42]

Nikil Wale, Ian A Watson, and George Karypis. 2008. Comparison of descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems 14, 3 (2008), 347–375.

Digital Library

[43]

Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, and Bryan Hooi. 2021. CurGraph: Curriculum Learning for Graph Classification. In TheWebConf. 1238–1248.

[44]

Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, and Bryan Hooi. 2021. Mixup for Node and Graph Classification. In TheWebConf. 3663–3674.

[45]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE TNNLS 32, 1 (2020), 4–24.

[46]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks?. In ICLR.

[47]

Pinar Yanardag and SVN Vishwanathan. 2015. Deep graph kernels. In KDD. 1365–1374.

[48]

Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. 2021. On Size Generalization in Graph Neural Networks. https://openreview.net/forum?id=9p2CltauWEY

[49]

Jianwen Yin, Chenghao Liu, Weiqing Wang, Jianling Sun, and Steven CH Hoi. 2020. Learning transferrable parameters for long-tailed sequential user behavior modeling. In KDD. 359–367.

[50]

Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018. Hierarchical Graph Representation Learning with Differentiable Pooling. In NeurIPS. 4800–4810.

[51]

Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI.

[52]

Tianxiang Zhao, Xiang Zhang, and Suhang Wang. 2021. GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks. In WSDM. 833–841.

[53]

Dawei Zhou, Jingrui He, Hongxia Yang, and Wei Fan. 2018. Sparc: Self-paced network representation for few-shot rare category characterization. In KDD. 2807–2816.

Digital Library

Cited By

Xu WWang PZhao ZWang BWang XWang YChua TNgo CKa-Wei Lee RKumar RLauw H(2024)When Imbalance Meets Imbalance: Structure-driven Learning for Imbalanced Graph ClassificationProceedings of the ACM on Web Conference 202410.1145/3589334.3645629(905-913)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645629
Tang HLiang XEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Where to Find Fascinating Inter-Graph Supervision: Imbalanced Graph Classification with Kernel Information BottleneckProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612039(3240-3249)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612039
Xiang TLi AJi YLi DSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Knowledge Based Prohibited Item Detection on Heterogeneous Risk GraphsProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599852(5260-5269)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599852
Show More Cited By

Index Terms

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Long-tailed graph neural networks via graph structure learning for node classification
Abstract
Long-tailed methods have gained increasing attention and achieved excellent performance due to the long-tailed distribution in graphs, i.e., many small-degree tail nodes have limited structural connectivity. However, real-world graphs are ...
RAHNet: Retrieval Augmented Hybrid Network for Long-tailed Graph Classification
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Graph classification is a crucial task in many real-world multimedia applications, where graphs can represent various multimedia data types such as images, videos, and social networks. Previous efforts have applied graph neural networks (GNNs) in ...
Streaming Graph Neural Networks
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Graphs are used to model pairwise relations between entities in many real-world scenarios such as social networks. Graph Neural Networks(GNNs) have shown their superior ability in learning representations for graph structured data, which leads to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

the Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funds

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
413
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)8

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu WWang PZhao ZWang BWang XWang YChua TNgo CKa-Wei Lee RKumar RLauw H(2024)When Imbalance Meets Imbalance: Structure-driven Learning for Imbalanced Graph ClassificationProceedings of the ACM on Web Conference 202410.1145/3589334.3645629(905-913)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645629
Tang HLiang XEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Where to Find Fascinating Inter-Graph Supervision: Imbalanced Graph Classification with Kernel Information BottleneckProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612039(3240-3249)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612039
Xiang TLi AJi YLi DSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Knowledge Based Prohibited Item Detection on Heterogeneous Risk GraphsProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599852(5260-5269)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599852
Lin JWan YXu JQi X(2023)Long-tailed graph neural networks via graph structure learning for node classificationApplied Intelligence10.1007/s10489-023-04534-353:17(20206-20222)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s10489-023-04534-3
Cui HLu ZLi PYang CAl Hasan MXiong L(2022)On Positional and Structural Node Features for Graph Neural Networks on Non-attributed GraphsProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557661(3898-3902)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557661

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents