research-article

Public Access

Dual Space Graph Contrastive Learning

Authors:

Guandong XuAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1238 - 1247

https://doi.org/10.1145/3485447.3512211

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Unsupervised graph representation learning has emerged as a powerful tool to address real-world problems and achieves huge success in the graph learning domain. Graph contrastive learning is one of the unsupervised graph representation learning methods, which recently attracts attention from researchers and has achieved state-of-the-art performances on various tasks. The key to the success of graph contrastive learning is to construct proper contrasting pairs to acquire the underlying structural semantics of the graph. However, this key part is not fully explored currently, most of the ways generating contrasting pairs focus on augmenting or perturbating graph structures to obtain different views of the input graph. But such strategies could degrade the performances via adding noise into the graph, which may narrow down the field of the applications of graph contrastive learning. In this paper, we propose a novel graph contrastive learning method, namely Dual Space Graph Contrastive (DSGC) Learning, to conduct graph contrastive learning among views generated in different spaces including the hyperbolic space and the Euclidean space. Since both spaces have their own advantages to represent graph data in the embedding spaces, we hope to utilize graph contrastive learning to bridge the spaces and leverage advantages from both sides. The comparison experiment results show that DSGC achieves competitive or better performances among all the datasets. In addition, we conduct extensive experiments to analyze the impact of different graph encoders on DSGC, giving insights about how to better leverage the advantages of contrastive learning between different spaces.

References

[1]

Aaron B. Adcock, Blair D. Sullivan, and Michael W. Mahoney. 2013. Tree-Like Structure in Large Social and Information Networks. In 2013 IEEE 13th International Conference on Data Mining, Dallas, TX, USA, December 7-10, 2013, Hui Xiong, George Karypis, Bhavani M. Thuraisingham, Diane J. Cook, and Xindong Wu (Eds.). IEEE Computer Society, 1–10. https://doi.org/10.1109/ICDM.2013.77

[2]

Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. 2013. Density-Based Clustering Based on Hierarchical Density Estimates. In Advances in Knowledge Discovery and Data Mining, Jian Pei, Vincent S. Tseng, Longbing Cao, Hiroshi Motoda, and Guandong Xu (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 160–172.

[3]

James W. Cannon, William J. Floyd, Richard Kenyon, Walter, and R. Parry. 1997. Hyperbolic geometry. In In Flavors of geometry. University Press, 59–115.

[4]

Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. 2019. Hyperbolic Graph Convolutional Neural Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 4869–4880. https://proceedings.neurips.cc/paper/2019/hash/0415740eaa4d9decbc8da001d3fd805f-Abstract.html

[5]

Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018. PME: Projected Metric Embedding on Heterogeneous Networks for Link Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, Yike Guo and Faisal Farooq (Eds.). ACM, 1177–1186. https://doi.org/10.1145/3219819.3219986

Digital Library

[6]

Thomas M. Cover and Peter E. Hart. 1967. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13, 1 (1967), 21–27. https://doi.org/10.1109/TIT.1967.1053964

Digital Library

[7]

David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (Eds.). 2224–2232. https://proceedings.neurips.cc/paper/2015/hash/f9be311e65d81a9ad8150a60844bb94c-Abstract.html

[8]

Naicheng Guo, Xiaolei Liu, Shaoshuai Li, Qiongxu Ma, Yunan Zhao, Bing Han, Lin Zheng, Kai-Xin Gao, and Xiaobo Guo. 2021. HCGR: Hyperbolic Contrastive Graph Representation Learning for Session-based Recommendation. CoRR abs/2107.05366(2021). arXiv:2107.05366https://arxiv.org/abs/2107.05366

[9]

William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 1024–1034. https://proceedings.neurips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html

[10]

Kaveh Hassani and Amir Hosein Khas Ahmadi. 2020. Contrastive Multi-View Representation Learning on Graphs. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event(Proceedings of Machine Learning Research, Vol. 119). PMLR, 4116–4126. http://proceedings.mlr.press/v119/hassani20a.html

[11]

R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=Bklr3j0cKX

[12]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=SJU4ayYgl

[13]

Nils M. Kriege and Petra Mutzel. 2012. Subgraph Matching Kernels for Attributed Graphs. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26 - July 1, 2012. icml.cc / Omnipress. http://icml.cc/2012/papers/542.pdf

[14]

Anchen Li, Bo Yang, Hongxu Chen, and Guandong Xu. 2021. Hyperbolic Neural Collaborative Recommender. CoRR abs/2104.07414(2021). arXiv:2104.07414https://arxiv.org/abs/2104.07414

[15]

Yicong Li, Hongxu Chen, Xiangguo Sun, Zhenchao Sun, Lin Li, Lizhen Cui, Philip S. Yu, and Guandong Xu. 2021. Hyperbolic Hypergraphs for Sequential Recommendation. CoRR abs/2108.08134(2021). arXiv:2108.08134https://arxiv.org/abs/2108.08134

[16]

Yadan Luo, Zi Huang, Hongxu Chen, Yang Yang, and Mahsa Baktashmotlagh. 2020. Interpretable Signed Link Prediction with Signed Infomax Hyperbolic Graph. CoRR abs/2011.12517(2020). arXiv:2011.12517https://arxiv.org/abs/2011.12517

[17]

Maximilian Nickel and Douwe Kiela. 2017. Poincaré Embeddings for Learning Hierarchical Representations. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 6338–6347. https://proceedings.neurips.cc/paper/2017/hash/59dfa2df42d9e3d41f5b02bfc32229dd-Abstract.html

[18]

Maximilian Nickel and Douwe Kiela. 2017. Poincaré Embeddings for Learning Hierarchical Representations. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 6338–6347. https://proceedings.neurips.cc/paper/2017/hash/59dfa2df42d9e3d41f5b02bfc32229dd-Abstract.html

[19]

Zhen Peng, Yixiang Dong, Minnan Luo, Xiao-Ming Wu, and Qinghua Zheng. 2020. Self-Supervised Graph Representation Learning via Global Context Prediction. CoRR abs/2003.01604(2020). arXiv:2003.01604https://arxiv.org/abs/2003.01604

[20]

Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. 2020. Graph Representation Learning via Graphical Mutual Information Maximization. In WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 259–270. https://doi.org/10.1145/3366423.3380112

Digital Library

[21]

Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (Eds.). ACM, 1150–1160. https://doi.org/10.1145/3394486.3403168

Digital Library

[22]

Benedek Rozemberczki and Rik Sarkar. 2020. Fast Sequence-Based Embedding with Diffusion Graphs. CoRR abs/2001.07463(2020). arxiv:2001.07463https://arxiv.org/abs/2001.07463

[23]

Wenzhuo Song, Hongxu Chen, Xueyan Liu, Hongzhe Jiang, and Shengsheng Wang. 2021. Hyperbolic node embedding for signed networks. Neurocomputing 421(2021), 329–339. https://doi.org/10.1016/j.neucom.2020.10.008

[24]

Rianne van den Berg, Thomas N. Kipf, and Max Welling. 2017. Graph Convolutional Matrix Completion. CoRR abs/1706.02263(2017). arXiv:1706.02263http://arxiv.org/abs/1706.02263

[25]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rJXMpikCZ

[26]

Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=rklz9iAcKQ

[27]

Yingheng Wang, Yaosen Min, Xin Chen, and Ji Wu. 2021. Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia (Eds.). ACM / IW3C2, 2921–2933. https://doi.org/10.1145/3442381.3449786

Digital Library

[28]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=ryGs6iA5Km

[29]

Pinar Yanardag and S. V. N. Vishwanathan. 2015. Deep Graph Kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, August 10-13, 2015, Longbing Cao, Chengqi Zhang, Thorsten Joachims, Geoffrey I. Webb, Dragos D. Margineantu, and Graham Williams (Eds.). ACM, 1365–1374. https://doi.org/10.1145/2783258.2783417

Digital Library

[30]

Haoran Yang, Hongxu Chen, Lin Li, Philip S. Yu, and Guandong Xu. 2021. Hyper Meta-Path Contrastive Learning for Multi-Behavior Recommendation. CoRR abs/2109.02859(2021). arXiv:2109.02859https://arxiv.org/abs/2109.02859

[31]

Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/hash/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html

[32]

Sixiao Zhang, Hongxu Chen, Xiao Ming, Lizhen Cui, Hongzhi Yin, and Guandong Xu. 2021. Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems. CoRR abs/2105.08908(2021). arxiv:2105.08908https://arxiv.org/abs/2105.08908

[33]

Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph Contrastive Learning with Adaptive Augmentation. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia (Eds.). ACM / IW3C2, 2069–2080. https://doi.org/10.1145/3442381.3449802

Digital Library

Cited By

Jiang ZGong ZDai XZhang HDing PShen C(2024)Deep graph contrastive learning model for drug-drug interaction predictionPLOS ONE10.1371/journal.pone.030479819:6(e0304798)Online publication date: 17-Jun-2024
https://doi.org/10.1371/journal.pone.0304798
Yao HCao NRussell KChang DFrieder OFineman J(2024)Self-Supervised Representation Learning on Electronic Health Records with Graph Kernel InfomaxACM Transactions on Computing for Healthcare10.1145/36486955:2(1-28)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3648695
Tang JYang YWei WShi LSu LCheng SYin DHuang CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)GraphGPT: Graph Instruction Tuning for Large Language ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657775(491-500)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657775
Show More Cited By

Index Terms

Dual Space Graph Contrastive Learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Cross-view graph contrastive learning with hypergraph
Abstract
Graph contrastive learning (GCL) provides a new perspective to alleviate the reliance on labeled data for graph representation learning. Recent efforts on GCL leverage various graph augmentation strategies, i.e., node dropping and edge masking, ...
Highlights
- We proposed that hypergraphs are used as a paradigm to enhance graph contrastive learning.
- We propose a novel diffusion model-based fusion mechanism that aligns the positive examples.
- Our experimental results all exceed existing ...
Learning to solve graph metric dimension problem based on graph contrastive learning
Abstract
Deep learning has been widely used to solve graph and combinatorial optimization problems. However, proper model deployment is critical for training a model and solving all problems. Existing frameworks mainly use reinforcement learning to learn ...
Line graph contrastive learning for node classification
Abstract
Existing graph contrastive learning methods often rely on differences in node features within subgraphs, lacking effective capture of the global structural information of the graph. To address this issue, we propose a novel graph contrastive ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
1,007
Total Downloads

Downloads (Last 12 months)440
Downloads (Last 6 weeks)56

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang ZGong ZDai XZhang HDing PShen C(2024)Deep graph contrastive learning model for drug-drug interaction predictionPLOS ONE10.1371/journal.pone.030479819:6(e0304798)Online publication date: 17-Jun-2024
https://doi.org/10.1371/journal.pone.0304798
Yao HCao NRussell KChang DFrieder OFineman J(2024)Self-Supervised Representation Learning on Electronic Health Records with Graph Kernel InfomaxACM Transactions on Computing for Healthcare10.1145/36486955:2(1-28)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3648695
Tang JYang YWei WShi LSu LCheng SYin DHuang CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)GraphGPT: Graph Instruction Tuning for Large Language ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657775(491-500)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657775
Qiu ZMa CWu JYang JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)An Efficient Automatic Meta-Path Selection for Social Event Detection via Hyperbolic SpaceProceedings of the ACM on Web Conference 202410.1145/3589334.3645526(2519-2529)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645526
Gao YLi XHui Y(2024)Rethinking Graph Contrastive Learning: An Efficient Single-View Approach via Instance DiscriminationIEEE Transactions on Multimedia10.1109/TMM.2023.331326726(3616-3625)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3313267
Cui JChai HYang XDing YFang BLiao Q(2024)SGCL: Semantic-aware Graph Contrastive Learning with Lipschitz Graph Augmentation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00235(3028-3041)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00235
Ju WFang ZGu YLiu ZLong QQiao ZQin YShen JSun FXiao ZYang JYuan JZhao YWang YLuo XZhang M(2024)A Comprehensive Survey on Deep Graph Representation LearningNeural Networks10.1016/j.neunet.2024.106207173(106207)Online publication date: May-2024
https://doi.org/10.1016/j.neunet.2024.106207
Wang HZhang WMa X(2024)Contrastive and adversarial regularized multi-level representation learning for incomplete multi-view clusteringNeural Networks10.1016/j.neunet.2024.106102172:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106102
Adjeisah MZhu XXu HAyall T(2024)Graph Contrastive Multi-view Learning: A Pre-training Framework for Graph ClassificationKnowledge-Based Systems10.1016/j.knosys.2024.112112299(112112)Online publication date: Sep-2024
https://doi.org/10.1016/j.knosys.2024.112112
Wei YDuan MZhou HJia ZGao ZWang L(2024)Towards multimodal sarcasm detection via label-aware graph contrastive learning with back-translation augmentationKnowledge-Based Systems10.1016/j.knosys.2024.112109300(112109)Online publication date: Sep-2024
https://doi.org/10.1016/j.knosys.2024.112109
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents