Learning Graph Neural Networks using Exact Compression
Authors: 
Jeroen Bollen, Jasper Steegmans, 
Jan Van Den Bussche, 
Stijn VansummerenAuthors Info & Claims
Jeroen Bollen, Jasper Steegmans, Jan Van Den Bussche, Stijn VansummerenAuthors Info & ClaimsArticle No.: 8, Pages 1 - 9
Abstract
Graph Neural Networks (GNNs) are a form of deep learning that enable a wide range of machine learning applications on graph-structured data. The learning of GNNs, however, is known to pose challenges for memory-constrained devices such as GPUs. In this paper, we study exact compression as a way to reduce the memory requirements of learning GNNs on large graphs. In particular, we adopt a formal approach to compression and propose a methodology that transforms GNN learning problems into provably equivalent compressed GNN learning problems. In a preliminary experimental evaluation, we give insights into the compression ratios that can be obtained on real-world graphs and apply our methodology to an existing GNN benchmark.
References
[1]
Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. MIT Press.
[2]
Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan-Pablo Silva. 2020. The Expressive Power of Graph Neural Networks as a Query Language. ACM SIGMOD Record 49, 2 (2020), 6--17.
[3]
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018. Relational Inductive Biases, Deep Learning, and Graph Networks. arXiv:arXiv:1806.01261
[4]
Jeroen Bollen, Jasper Steegmans, Jan Van den Bussche, and Stijn Vansummeren. 2023. Learning Graph Neural Networks using Exact Compression (Extended Version). Technical Report. http://arxiv.org/abs/2304.14793.
[5]
A. Cardon and M. Crochemore. 1982. Partitioning a Graph in O (|A| log2 |V|). Theoretical Computer Science 19, 1 (July 1982), 85--98.
[6]
Jie Chen, Tengfei Ma, and Cao Xiao. 2018. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In ICLR. OpenReview.net.
[7]
Chenhui Deng, Zhiqiang Zhao, Yongyu Wang, Zhiru Zhang, and Zhuo Feng. [n. d.]. GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding. In ICLR 2020. OpenReview.net.
[8]
Floris Geerts, Jasper Steegmans, and Jan Van den Bussche. 2022. On the Expressive Power of Message-Passing Neural Networks as Global Feature Map Transformers. In Foundations of Information and Knowledge Systems (LNCS). Springer, 20--34.
[9]
Alessandro Generale, Till Blume, and Michael Cochez. 2022. Scaling R-GCN Training with Graph Summarization. In Companion Proceedings of the Web Conference 2022 (WWW '22). ACM, 1073--1082.
[10]
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. [n. d.]. Neural Message Passing for Quantum Chemistry. In ICML 2017. JMLR.org, 1263--1272.
[11]
Martin Grohe. [n. d.]. The Logic of Graph Neural Networks. In LICS '21. ACM, 1--17.
[12]
William L. Hamilton. 2020. Graph Representation Learning. Morgan & Claypool Publishers.
[13]
William L. Hamilton, Rex Ying, and Jure Leskovec. [n. d.]. Inductive Representation Learning on Large Graphs. In NIPS 2017. Curran Associates Inc., 1025--1035.
[14]
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 22118--22133.
[15]
Wenbing Huang, Tong Zhang, Yu Rong, and Junzhou Huang. [n. d.]. Adaptive Sampling towards Fast Graph Representation Learning. In NIPS 2018. Curran Associates Inc., 4563--4572.
[16]
Jure Leskovec and Andrej Krevl. [n. d.]. SNAP Datasets: Stanford Large Network Dataset Collection.
[17]
Jiongqian Liang, Saket Gurukar, and Srinivasan Parthasarathy. 2021. MILE: A Multi-Level Framework for Scalable Graph Embedding. Proceedings of the International AAAI Conference on Web and Social Media 15 (May 2021), 361--372.
[18]
Ningyi Liao, Dingheng Mo, Siqiang Luo, Xiang Li, and Pengcheng Yin. 2022. SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization. PVLDB 15, 11 (2022), 3240--3248.
[19]
Haiyang Lin, Mingyu Yan, Xiaocheng Yang, Mo Zou, Wenming Li, Xiaochun Ye, and Dongrui Fan. 2022. Characterizing and Understanding Distributed GNN Training on GPUs. IEEE Computer Architecture Letters 21, 1 (2022), 21--24.
[20]
Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. 2019. Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks. AAAI 33, 01 (2019), 4602--4609.
[21]
Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. 2022. Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks. PVLDB 15, 9 (2022), 1937--1950.
[22]
Guillaume Salha, Romain Hennequin, Viet-Anh Tran, and Michalis Vazirgiannis. 2019. A Degeneracy Framework for Scalable Graph Autoencoders. In IJCAI. ijcai.org, 3353--3359.
[23]
Qiange Wang, Yanfeng Zhang, Hao Wang, Chaoyi Chen, Xiaodong Zhang, and Ge Yu. [n. d.]. NeutronStar: Distributed GNN Training with Hybrid Dependency Management. In SIGMOD 2022. ACM, 1301--1315.
[24]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2021. A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2021), 4--24.
[25]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. [n. d.]. How Powerful Are Graph Neural Networks?. In ICLR 2019. OpenReview.net.
[26]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. [n. d.]. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In KDD 2018. ACM, 974--983.
[27]
Binhang Yuan, Cameron R. Wolfe, Chen Dun, Yuxin Tang, Anastasios Kyrillidis, and Chris Jermaine. 2022. Distributed Learning of Fully Connected Neural Networks Using Independent Subnet Training. PVLDB 15, 8 (2022), 1581--1590.
[28]
Chenguang Zheng, Hongzhi Chen, Yuxuan Cheng, Zhezheng Song, Yifan Wu, Changji Li, James Cheng, Hao Yang, and Shuai Zhang. 2022. ByteGNN: Efficient Graph Neural Network Training at Large Scale. PVLDB 15, 6 (2022), 1228--1242.
[29]
Zulun Zhu, Jiaying Peng, Jintang Li, Liang Chen, Qi Yu, and Siqiang Luo. 2022. Spiking Graph Convolutional Networks. In IJCAI, Vol. 3. 2434--2440.
Index Terms
- Learning Graph Neural Networks using Exact Compression
Recommendations
On Graph Learning with Neural Networks
Machine Learning, Optimization, and Data ScienceAbstractGraphs are ideal for modeling natural systems where relations may be intrinsic among data objects. With massive data available, learning graph models from data has become potentially feasible as well as necessary. Yet from the traditional machine ...
Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks
Learning and Intelligent OptimizationAbstractThe graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a ...
Toward the analysis of graph neural networks
ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging ResultsGraph Neural Networks (GNNs) have recently emerged as an effective framework for representing and analyzing graph-structured data. GNNs have been applied to many real-world problems such as knowledge graph analysis, social networks recommendation, and ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Fund for Scientific Research Flanders (FWO)
- Bijzonder Onderzoeksfonds (BOF) of Hasselt University
Conference
GRADES & NDA '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 29 of 61 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 146Total Downloads
- Downloads (Last 12 months)111
- Downloads (Last 6 weeks)8
Reflects downloads up to 02 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in