Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3442381.3449884acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Hashing-Accelerated Graph Neural Networks for Link Prediction

Published: 03 June 2021 Publication History

Abstract

Networks are ubiquitous in the real world. Link prediction, as one of the key problems for network-structured data, aims to predict whether there exists a link between two nodes. The traditional approaches are based on the explicit similarity computation between the compact node representation by embedding each node into a low-dimensional space. In order to efficiently handle the intensive similarity computation in link prediction, the hashing technique has been successfully used to produce the node representation in the Hamming space. However, the hashing-based link prediction algorithms face accuracy loss from the randomized hashing techniques or inefficiency from the learning to hash techniques in the embedding process. Currently, the Graph Neural Network (GNN) framework has been widely applied to the graph-related tasks in an end-to-end manner, but it commonly requires substantial computational resources and memory costs due to massive parameter learning, which makes the GNN-based algorithms impractical without the help of a powerful workhorse. In this paper, we propose a simple and effective model called #GNN, which balances the trade-off between accuracy and efficiency. #GNN is able to efficiently acquire node representation in the Hamming space for link prediction by exploiting the randomized hashing technique to implement message passing and capture high-order proximity in the GNN framework. Furthermore, we characterize the discriminative power of #GNN in probability. The extensive experimental results demonstrate that the proposed #GNN algorithm achieves accuracy comparable to the learning-based algorithms and outperforms the randomized algorithm, while running significantly faster than the learning-based algorithms. Also, the proposed algorithm shows excellent scalability on a large-scale network with the limited resources.

References

[1]
Lada A Adamic and Eytan Adar. 2003. Friends and Neighbors on the Web. Social networks 25, 3 (2003), 211–230.
[2]
Charu C Aggarwal. 2011. On Classification of Graph Streams. In SDM. 652–663.
[3]
Andrei Z Broder, Moses Charikar, Alan M Frieze, and Michael Mitzenmacher. 1998. Min-wise Independent Permutations. In STOC. 327–336.
[4]
Moses S Charikar. 2002. Similarity Estimation Techniques from Rounding Algorithms. In STOC. 380–388.
[5]
David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In NIPS. 2224–2232.
[6]
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In ICML. 1263–1272.
[7]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In KDD. 855–864.
[8]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS. 1024–1034.
[9]
Qing-Yuan Jiang and Wu-Jun Li. 2015. Scalable Graph Hashing with Feature Transformation. In AAAI. 2248–2254.
[10]
Steven Kearnes, Kevin McCloskey, Marc Berndl, Vijay Pande, and Patrick Riley. 2016. Molecular Graph Convolutions: Moving beyond Fingerprints. Journal of computer-aided molecular design 30, 8 (2016), 595–608.
[11]
Thomas N Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[12]
Leonid Aryeh Kontorovich, Kavita Ramanan, 2008. Concentration Inequalities for Dependent Random Variables via the Martingale Method. The Annals of Probability 36, 6 (2008), 2126–2158.
[13]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. Computer 42, 8 (2009), 30–37.
[14]
Tao Lei, Wengong Jin, Regina Barzilay, and Tommi Jaakkola. 2017. Deriving Neural Architectures from Sequence and Graph Kernels. In ICML. 2024–2033.
[15]
Jure Leskovec and Julian J Mcauley. 2012. Learning to Discover Social Circles in Ego Networks. In NIPS. 539–547.
[16]
Bin Li, Xingquan Zhu, Lianhua Chi, and Chengqi Zhang. 2012. Nested Subtree Hash Kernels for Large-scale Graph Classification over Streams. In ICDM. 399–408.
[17]
Jundong Li, Xia Hu, Jiliang Tang, and Huan Liu. 2015. Unsupervised Streaming Feature Selection in Social Media. In CIKM. 1041–1050.
[18]
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated Graph Sequence Neural Networks. In ICLR.
[19]
Defu Lian, Kai Zheng, Vincent W Zheng, Yong Ge, Longbing Cao, Ivor W Tsang, and Xing Xie. 2018. High-order Proximity Preserving Information Network Hashing. In SIGKDD. 1744–1753.
[20]
David Liben-Nowell and Jon Kleinberg. 2007. The Link-prediction Problem for Social Networks. Journal of the Association for Information Science and Technology 58, 7(2007), 1019–1031.
[21]
Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. 2014. Discrete Graph Hashing. In NIPS. 3419–3427.
[22]
Gurmeet Singh Manku, Arvind Jain, and Anish Das Sarma. 2007. Detecting Near-duplicates for Web Crawling. In WWW. 141–150.
[23]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and Their Compositionality. In NIPS. 3111–3119.
[24]
Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. 2019. Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks. In AAAI. 4602–4609.
[25]
Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2015. A Review of Relational Machine Learning for Knowledge Graphs. Proc. IEEE 104, 1 (2015), 11–33.
[26]
Tolutola Oyetunde, Muhan Zhang, Yixin Chen, Yinjie Tang, and Cynthia Lo. 2017. Boostgapfill: Improving the Fidelity of Metabolic Network Reconstructions through Integrated Constraint and Pattern-based Methods. Bioinformatics 33, 4 (2017), 608–611.
[27]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In KDD. 701–710.
[28]
Jiezhong Qiu, Yuxiao Dong, Hao Ma, Jian Li, Kuansan Wang, and Jie Tang. 2018. Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec. In WSDM. 459–467.
[29]
Ruslan Salakhutdinov and Geoffrey Hinton. 2009. Semantic Hashing. International Journal of Approximate Reasoning 50, 7 (2009), 969–978.
[30]
Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. 2017. Quantum-chemical Insights from Deep Tensor Neural Networks. Nature communications 8, 1 (2017), 1–8.
[31]
Enya Shen, Zhidong Cao, Changqing Zou, and Jianmin Wang. 2018. Flexible Attributed Network Embedding. arxiv:1811.10789 [cs.SI]
[32]
Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-Lehman Graph Kernels.Journal of Machine Learning Research 12, 9 (2011).
[33]
Xiaoshuang Shi, Fuyong Xing, Kaidi Xu, Manish Sapkota, and Lin Yang. 2017. Asymmetric Discrete Graph Hashing. In AAAI. 2541–2547.
[34]
Qiaoyu Tan, Ninghao Liu, Xing Zhao, Hongxia Yang, Jingren Zhou, and Xia Hu. 2020. Learning to Hash with Graph Neural Networks for Recommender Systems. In WWW. 1988–1998.
[35]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale Information Network Embedding. In WWW. 1067–1077.
[36]
Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. ArnetMiner: Extraction and Mining of Academic Social Networks. In KDD. 990–998.
[37]
Cunchao Tu, Han Liu, Zhiyuan Liu, and Maosong Sun. 2017. Cane: Context-Aware Network Embedding for Relation Modeling. In ACL, Vol. 1. 1722–1731.
[38]
Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature Hashing for Large Scale Multitask Learning. In ICML. 1113–1120.
[39]
Boris Weisfeiler and Andrei A Lehman. 1968. A Reduction of a Graph to a Canonical Form and an Algebra Arising During this Reduction. Nauchno-Technicheskaya Informatsia 2, 9 (1968), 12–16.
[40]
Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral Hashing. In NIPS. 1753–1760.
[41]
Wei Wu, Bin Li, Ling Chen, Junbin Gao, and Chengqi Zhang. 2020. A Review for Weighted MinHash Algorithms. IEEE Transactions on Knowledge and Data Engineering (2020), 1–1.
[42]
Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. 2016. Canonical Consistent Weighted Sampling for Real-Value Weighted Min-Hash. In ICDM. 1287–1292.
[43]
Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. 2017. Consistent Weighted Sampling Made More Practical. In WWW. 1035–1043.
[44]
Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. 2018. Efficient Attributed Network Embedding via Recursive Randomized Hashing. In IJCAI-18. 2861–2867.
[45]
Wei Wu, Bin Li, Ling Chen, Chengqi Zhang, and S Yu Philip. 2018. Improved Consistent Weighted Sampling Revisited. IEEE Transactions on Knowledge and Data Engineering 31, 12(2018), 2332–2345.
[46]
Wei Wu, Bin Li, Ling Chen, Xingquan Zhu, and Chengqi Zhang. 2018. K-Ary Tree Hashing for Fast Graph Classification. IEEE Transactions on Knowledge and Data Engineering 30, 5(2018), 936–949.
[47]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In ICLR.
[48]
Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. 2015. Network Representation Learning with Rich Text Information. In IJCAI. 2111–2117.
[49]
Dingqi Yang, Paolo Rosso, Bin Li, and Philippe Cudre-Mauroux. 2019. NodeSketch: Highly-Efficient Graph Embeddings via Recursive Sketching. In KDD. 1162–1172.
[50]
Hong Yang, Shirui Pan, Ling Chen, Chuan Zhou, and Peng Zhang. 2019. Low-Bit Quantization for Attributed Network Representation Learning. In IJCAI. 4047–4053.
[51]
Hong Yang, Shirui Pan, Peng Zhang, Ling Chen, Defu Lian, and Chengqi Zhang. 2018. Binarized Attributed Network Embedding. In ICDM. 1476–1481.
[52]
Jiaxuan You, Rex Ying, and Jure Leskovec. 2019. Position-aware Graph Neural Networks. In ICML. 7134–7143.
[53]
Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2016. Homophily, Structure, and Content Augmented Network Representation Learning. In ICDM. 609–618.
[54]
Li Zhang, Dan Xu, Anurag Arnab, and Philip HS Torr. 2020. Dynamic Graph Message Passing Networks. In CVPR. 3726–3735.
[55]
Muhan Zhang and Yixin Chen. 2017. Weisfeiler-Lehman Neural Machine for Link Prediction. In KDD. 575–583.
[56]
Muhan Zhang and Yixin Chen. 2018. Link Prediction based on Graph Neural Networks. In NeurIPS. 5165–5175.
[57]
Yang Zhou, Zachary While, and Evangelos Kalogerakis. 2019. SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation. In ICCV. 7384–7392.

Cited By

View all
  • (2025)Time- and Space-Efficiently Sketching Billion-Scale Attributed NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350825637:2(966-978)Online publication date: Feb-2025
  • (2024)PACER: Accelerating Distributed GNN Training Using Communication-Efficient Partition Refinement and CachingProceedings of the ACM on Networking10.1145/36978052:CoNEXT4(1-18)Online publication date: 25-Nov-2024
  • (2024)Bridging and Compressing Feature and Semantic Spaces for Robust Graph Neural Networks: An Information Theory PerspectiveProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671870(4571-4582)Online publication date: 25-Aug-2024
  • Show More Cited By
  1. Hashing-Accelerated Graph Neural Networks for Link Prediction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '21: Proceedings of the Web Conference 2021
    April 2021
    4054 pages
    ISBN:9781450383127
    DOI:10.1145/3442381
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 June 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Attributed Network
    2. Graph Neural Networks
    3. Hashing
    4. Link Prediction

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    WWW '21
    Sponsor:
    WWW '21: The Web Conference 2021
    April 19 - 23, 2021
    Ljubljana, Slovenia

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)90
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 31 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Time- and Space-Efficiently Sketching Billion-Scale Attributed NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350825637:2(966-978)Online publication date: Feb-2025
    • (2024)PACER: Accelerating Distributed GNN Training Using Communication-Efficient Partition Refinement and CachingProceedings of the ACM on Networking10.1145/36978052:CoNEXT4(1-18)Online publication date: 25-Nov-2024
    • (2024)Bridging and Compressing Feature and Semantic Spaces for Robust Graph Neural Networks: An Information Theory PerspectiveProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671870(4571-4582)Online publication date: 25-Aug-2024
    • (2024)MPSketch: Message Passing Networks via Randomized Hashing for Efficient Attributed Network EmbeddingIEEE Transactions on Cybernetics10.1109/TCYB.2023.324376354:5(2941-2954)Online publication date: May-2024
    • (2024)In-memory search with learning to hash based on resistive memory for recommendation accelerationnpj Unconventional Computing10.1038/s44335-024-00009-x1:1Online publication date: 1-Oct-2024
    • (2024)NLA-GNN: Non-local information aggregated graph neural network for heterogeneous graph embeddingPattern Recognition10.1016/j.patcog.2024.110940(110940)Online publication date: Sep-2024
    • (2024)Manufacturing resilience through disruption mitigation using attention-based consistently-attributed graph embedded decision support systemComputers & Industrial Engineering10.1016/j.cie.2024.110494197(110494)Online publication date: Nov-2024
    • (2024)Rcoco: contrastive collective link prediction across multiplex network in Riemannian spaceInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02118-215:9(3745-3767)Online publication date: 5-Apr-2024
    • (2023)Hessian-aware Quantized Node Embeddings for RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608826(757-762)Online publication date: 14-Sep-2023
    • (2023)From random-walks to graph-sprints: a low-latency node embedding framework on continuous-time dynamic graphsProceedings of the Fourth ACM International Conference on AI in Finance10.1145/3604237.3626842(176-184)Online publication date: 27-Nov-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media