A Caching-based Framework for Scalable Temporal Graph Neural Network Training
Article No.: 3, Pages 1 - 46
Abstract
Representation learning over dynamic graphs is critical for many real-world applications such as social network services and recommender systems. Temporal graph neural networks (T-GNNs) are powerful representation learning methods and have demonstrated remarkable effectiveness on continuous-time dynamic graphs. However, T-GNNs still suffer from high time complexity, which increases linearly with the number of timestamps and grows exponentially with the model depth, making them not scalable to large dynamic graphs. To address the limitations, we propose Orca, a novel framework that accelerates T-GNN training by caching and reusing intermediate embeddings. We design an optimal caching policy, named MRD, for the uniform cache replacement problem, where embeddings at different intermediate layers have identical dimensions and recomputation costs. MRD not only improves the efficiency of training T-GNNs by maximizing the number of cache hits but also reduces the approximation errors by avoiding keeping and reusing extremely stale embeddings. For the general cache replacement problem, where embeddings at different intermediate layers can have different dimensions and recomputation costs, we solve this NP-hard problem by presenting a novel two-stage framework with approximation guarantees on the achieved benefit of caching. Furthermore, we have developed profound theoretical analyses of the approximation errors introduced by reusing intermediate embeddings, providing a thorough understanding of the impact of our caching and reuse schemes on model outputs. We also offer rigorous convergence guarantees for model training, adding to the reliability and validity of our Orca framework. Extensive experiments have validated that Orca can obtain two orders of magnitude speedup over state-of-the-art T-GNNs while achieving higher precision on various dynamic graphs.
References
[1]
2024. AskUbuntu. Retrieved from http://snap.stanford.edu/data/sx-askubuntu.html
[2]
2024. SuperUser. Retrieved from http://snap.stanford.edu/data/sx-superuser.html
[3]
2024. Wiki-talk. Retrieved from http://snap.stanford.edu/data/wiki-talk-temporal.html
[4]
2024. Wikipedia Edit History Dump. Retrieved from https://meta.wikimedia.org/wiki/Data_dumps
[5]
Bilge Acun, Matthew Murphy, Xiaodong Wang, Jade Nie, Carole-Jean Wu, and Kim M. Hazelwood. 2021. Understanding training efficiency of deep learning recommendation models at scale. In Proceedings of theInternational Symposium on High-Performance Computer Architecture. IEEE, 802–814.
[6]
Susanne Albers, Sanjeev Arora, and Sanjeev Khanna. 1999. Page replacement for general caching problems. In Proceedings of the 10th annual ACM-SIAM symposium on Discrete algorithms. ACM/SIAM, 31–40.
[7]
Raghu Arghal, Eric Lei, and Shirin Saeedi Bidokhti. 2022. Robust graph neural networks via probabilistic lipschitz constraints. In Learning for Dynamics and Control Conference, L4DC 2022, 23-24 June 2022, Stanford University, Stanford, CA, USA (Proceedings of Machine Learning Research), Roya Firoozi, Negar Mehr, Esen Yel, Rika Antonova, Jeannette Bohg, Mac Schwager, and Mykel J. Kochenderfer (Eds.). Vol. 168, PMLR, 1073–1085. Retrieved from https://proceedings.mlr.press/v168/arghal22a.html
[8]
Amotz Bar-Noy, Reuven Bar-Yehuda, Ari Freund, Joseph Naor, and Baruch Schieber. 2000. A unified approach to approximating resource allocation and scheduling. In Proceedings of the 32nd annual ACM symposium on Theory of computing. ACM, 735–744.
[9]
Laszlo A. Belady. 1966. A study of replacement algorithms for virtual-storage computer. IBM Systems Journal 5, 2 (1966), 78–101.
[10]
Anant P. Bhardwaj, Souvik Bhattacherjee, Amit Chavan, Amol Deshpande, Aaron J. Elmore, Samuel Madden, and Aditya G. Parameswaran. 2015. DataHub: Collaborative data science & dataset version management at scale. In Proceedings of the Conference on Innovative Data Systems Research. Retrieved from www.cidrdb.org
[11]
Souvik Bhattacherjee, Amit Chavan, Silu Huang, Amol Deshpande, and Aditya G. Parameswaran. 2015. Principles of dataset versioning: Exploring the recreation/storage tradeoff. PVLDB 8, 12 (2015), 1346–1357.
[12]
Matthias Boehm, Arun Kumar, and Jun Yang. 2019. Data Management in Machine Learning Systems. Morgan & Claypool Publishers.
[13]
Jianfei Chen, Jun Zhu, and Le Song. 2018. Stochastic training of graph convolutional networks with variance reduction. In Proceedings of the International Conference on Machine Learning, Vol. 80. PMLR, 941–949.
[14]
Rada Chirkova and Jun Yang. 2012. Materialized Views. Foundations and Trends in Databases (TODS) 4, 4 (2012), 295–405.
[15]
Marek Chrobak and John Noga. 1998. LRU is better than FIFO. In Proceedings of the PODS. ACM/SIAM, 78–81.
[16]
Marek Chrobak, Gerhard J. Woeginger, Kazuhisa Makino, and Haifeng Xu. 2012. Caching Is hard - even in the fault model. Algorithmica 63, 4 (2012), 781–794.
[17]
Weilin Cong, Rana Forsati, Mahmut T. Kandemir, and Mehrdad Mahdavi. 2020. Minimal variance sampling with provable guarantees for fast training of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1393–1403.
[18]
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. 2023. Do we really need complicated model architectures for temporal networks? In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net. Retrieved from https://openreview.net/forum?id=ayPPc0SyLv1
[19]
Daniel Crankshaw, Xin Wang, Giulio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. 2017. Clipper: A low-latency online prediction serving system. In Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation. USENIX Association, 613–627.
[20]
Behrouz Derakhshan, Alireza Rezaei Mahdiraji, Ziawasch Abedjan, Tilmann Rabl, and Volker Markl. 2020. Optimizing machine learning workloads in collaborative environments. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. ACM, 1701–1716.
[21]
Behrouz Derakhshan, Alireza Rezaei Mahdiraji, Zoi Kaoudi, Tilmann Rabl, and Volker Markl. 2022. Materialization and reuse optimizations for production data science pipelines. In Proceedings of the 2022 International Conference on Management of Data. ACM, 1962–1976.
[22]
Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Jure Leskovec. 2021. GNNAutoScale: Scalable and expressive graph neural networks via historical embeddings. In Proceedings of the International Conference on Machine Learning, Vol. 139. 3294–3304.
[23]
Lukáš Folwarczný and Jirí Sgall. 2017. General caching is hard: Even with small pages. Algorithmica 79, 2 (2017), 319–339.
[24]
Arnaud Fréville. 2004. The multidimensional 0-1 knapsack problem: An overview. European Journal of Operational Research 155, 1 (2004), 1–21.
[25]
Fernando Gama, Joan Bruna, and Alejandro Ribeiro. 2020. Stability properties of graph neural networks. IEEE Transactions on Signal Process. 68 (2020), 5680–5695.
[26]
Shihong Gao, Yiming Li, Yanyan Shen, Yingxia Shao, and Lei Chen. 2024. ETC: Efficient training of temporal graph neural networks over large-scale dynamic graphs. PVLDB 17, 5 (2024), 1060–1072.
[27]
Shihong Gao, Yiming Li, Xin Zhang, Yanyan Shen, Yingxia Shao, and Lei Chen. 2024. SIMPLE: Efficient temporal graph neural network training at scale with dynamic data placement. Proceedings of the ACM on Management of Data 2, 3 (2024), 174.
[28]
Federico Girosi, Michael J. Jones, and Tomaso A. Poggio. 1995. Regularization theory and neural networks architectures. Neural Computation 7, 2 (1995), 219–269.
[29]
Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. 2020. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge Based System 187 (2020).
[30]
Palash Goyal, Sujit Rokka Chhetri, Ninareh Mehrabi, Emilio Ferrara, and Arquimedes Canedo. 2018. DynamicGEM: A library for dynamic graph embedding methods. arXiv:1811.10734. Retrieved from https://arxiv.org/abs/1811.10734
[31]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 855–864.
[32]
Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna R. Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. 2019. Variational graph recurrent neural networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. 10700–10710.
[33]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation learning on graphs: Methods and applications. IEEE Data Engineering Bulletin 40, 3 (2017), 52–74.
[34]
William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 1024–1034.
[35]
Boris Hanin. 2018. Which neural net architectures give rise to exploding and vanishing gradients?. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 580–589.
[36]
Eric N. Hanson. 1987. A performance analysis of view materialization strategies. In Proceedings of the SIGMOD. ACM Press, 440–453.
[37]
Sandy Irani. 2002. Page replacement with multi-size pages and applications to web caching. Algorithmica 33, 3 (2002), 384–409.
[38]
Kam-Chuen Jim, C. Lee Giles, and Bill G. Horne. 1996. An analysis of noise in recurrent neural networks: Convergence and generalization. IEEE Transactions on Neural Networks 7, 6 (1996), 1424–1438.
[39]
Theodore Johnson and Dennis E. Shasha. 1994. 2Q: A low overhead high performance buffer management replacement algorithm. In Proceedings of the 20th International Conference on Very Large Data Bases. Morgan Kaufmann, 439–450.
[40]
Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. NoScope: Optimizing deep CNN-based queries over video streams at scale. PVLDB 10, 11 (2017), 1586–1597.
[41]
Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv:1611.07308. Retrieved from https://arxiv.org/abs/1611.07308
[42]
Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1269–1278.
[43]
Yunseong Lee, Alberto Scolari, Byung-Gon Chun, Marco Domenico Santambrogio, Markus Weimer, and Matteo Interlandi. 2018. PRETZEL: Opening the black box of machine learning prediction serving systems. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation. USENIX Association, 611–626.
[44]
Haoyang Li and Lei Chen. 2021. Cache-based GNN system for dynamic graphs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. ACM, 937–946.
[45]
Yiming Li, Yanyan Shen, and Lei Chen. 2022. Camel: Managing data for efficient stream learning. In Proceedings of the 2022 International Conference on Management of Data. ACM, 1271–1285.
[46]
Yiming Li, Yanyan Shen, Lei Chen, and Mingxuan Yuan. 2023. Orca: Scalable temporal graph neural network training with theoretical guarantees. Proceedings of the ACM on Management of Data 1, 1 (2023), 52:1–52:27.
[47]
Yiming Li, Yanyan Shen, Lei Chen, and Mingxuan Yuan. 2023. Zebra: When temporal graph neural networks meet temporal personalized pagerank. PVLDB 16, 6 (2023), 1332–1345.
[48]
Yanxi Li, Zean Wen, Yunhe Wang, and Chang Xu. 2021. One-shot graph neural architecture search with dynamic search space. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 8510–8517.
[49]
Edo Liberty, Zohar S. Karnin, Bing Xiang, Laurence Rouesnel, Baris Coskun, Ramesh Nallapati, Julio Delgado, Amir Sadoughi, Yury Astashonok, Piali Das, Can Balioglu, Saswata Chakravarty, Madhav Jha, Philip Gautier, David Arpin, Tim Januschowski, Valentin Flunkert, Yuyang Wang, Jan Gasthaus, Lorenzo Stella, Syama Sundar Rangapuram, David Salinas, Sebastian Schelter, and Alex Smola. 2020. Elastic machine learning algorithms in amazon sagemaker. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. ACM, 731–737.
[50]
Dongsheng Luo, Wei Cheng, Wenchao Yu, Bo Zong, Jingchao Ni, Haifeng Chen, and Xiang Zhang. 2021. Learning to drop: Robust graph neural network via topological denoising. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. ACM, 779–787.
[51]
Gongxu Luo, Jianxin Li, Hao Peng, Carl J. Yang, Lichao Sun, Philip S. Yu, and Lifang He. 2021. Graph entropy guided node embedding dimension selection for graph neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence. Retrieved from ijcai.org, 2767–2774.
[52]
Yuhong Luo and Pan Li. 2022. Neighborhood-aware scalable temporal network representation learning. In Proceedings of the LOG(Proceedings of Machine Learning Research, Vol. 198). PMLR, 1.
[53]
Sedigheh Mahdavi, Shima Khoshraftar, and Aijun An. 2018. dynnode2vec: Scalable dynamic network embedding. In Proceedings of the 2018 IEEE International Conference on Big Data (Big Data). IEEE, 3762–3765.
[54]
Imene Mami and Zohra Bellahsene. 2012. A survey of view selection methods. SIGMOD Record 41, 1 (2012), 20–29.
[55]
Hui Miao, Ang Li, Larry S. Davis, and Amol Deshpande. 2017. ModelHub: Deep learning lifecycle management. In Proceedings of the IEEE International Conference on Data Engineering. IEEE Computer Society, 1393–1394.
[56]
Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, and Bin Cui. 2021. HET: Scaling out huge embedding model training via cache-enabled distributed framework. PVLDB 15, 2 (2021), 312–320.
[57]
Pierre Michaud. 2016. Some mathematical facts about optimal cache replacement. ACM Transactions on Architecture and Code Optimization 13, 4 (2016), 50:1–50:19.
[58]
Supun Nakandala and Arun Kumar. 2020. Vista: Optimized system for declarative feature transfer from deep CNNs at scale. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. ACM, 1685–1700.
[59]
Supun Nakandala and Arun Kumar. 2022. Nautilus: An optimized system for deep transfer learning over evolving training datasets. In Proceedings of the 2022 International Conference on Management of Data. ACM, 506–520.
[60]
S. Deepak Narayanan, Aditya Sinha, Prateek Jain, Purushottam Kar, and Sundararajan Sellamanickam. 2022. IGLU: Efficient GCN training via lazy updates. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.net. Retrieved from https://openreview.net/forum?id=5kq11Tl1z4
[61]
Babatounde Moctard Oloulade, Jianliang Gao, Jiamin Chen, Tengfei Lyu, and Raeed Al-Sabri. 2021. Graph neural architecture search: A survey. Tsinghua Science and Technology 27, 4 (2021), 692–708.
[62]
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, and Charles E. Leiserson. 2020. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence. 5363–5370.
[63]
Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. 2022. SANCUS: Staleness-aware communication-avoiding full-graph decentralized training in large-scale graph neural networks. PVLDB 15, 9 (2022), 1937–1950.
[64]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 701–710.
[65]
Arnab Phani, Benjamin Rath, and Matthias Boehm. 2021. LIMA: Fine-grained lineage tracing and reuse in machine learning systems. In Proceedings of the 2021 International Conference on Management of Data. ACM, 1426–1439.
[66]
Yijian Qin, Xin Wang, Zeyang Zhang, and Wenwu Zhu. 2021. Graph differentiable architecture search with structure learning. In Proceedings of the 35th International Conference on Neural Information Processing Systems. 16860–16872.
[67]
Hassan Ramchoun, Mohammed Amine Janati Idrissi, Youssef Ghanou, and Mohamed Ettaouil. 2016. Multilayer perceptron: Architecture optimization and training. International Journal of Interactive Multimedia and Artificial Intelligence 4, 1 (2016), 26–30.
[68]
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael M. Bronstein. 2020. Temporal graph networks for deep learning on dynamic graphs. arXiv:2006.10637. Retrieved from https://arxiv.org/abs/2006.10637
[69]
Benjamin Van Roy. 2007. A short proof of optimality for the MIN cache replacement algorithm. Information Process. Letter 102, 2-3 (2007), 72–73.
[70]
Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv:1609.04747. Retrieved from https://arxiv.org/abs/1609.04747
[71]
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. DySAT: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining. ACM, 519–527.
[72]
Andrew I. Schein, Alexandrin Popescul, Lyle H. Ungar, and David M. Pennock. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 253–260.
[73]
Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. 2018. Structured sequence modeling with graph convolutional recurrent networks. In Proceedings of the Neural Information Processing: 25th International Conference, ICONIP 2018, Vol. 11301. Springer, 362–373.
[74]
Zeyuan Shang, Emanuel Zgraggen, Benedetto Buratti, Ferdinand Kossmann, Philipp Eichmann, Yeounoh Chung, Carsten Binnig, Eli Upfal, and Tim Kraska. 2019. Democratizing data science through interactive curation of ML pipelines. In Proceedings of the 2019 International Conference on Management of Data. ACM, 1171–1188.
[75]
Zhihao Shi, Xize Liang, and Jie Wang. 2023. LMC: Fast training of GNNs via subgraph sampling with provable convergence. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net. Retrieved from https://openreview.net/forum?id=5VBBA91N6n
[76]
Joakim Skarding, Bogdan Gabrys, and Katarzyna Musial. 2021. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. IEEE Access 9 (2021), 79143–79168.
[77]
Evan R. Sparks, Shivaram Venkataraman, Tomer Kaftan, Michael J. Franklin, and Benjamin Recht. 2017. KeystoneML: Optimizing pipelines for large-scale advanced analytics. In 2017 IEEE 33rd International Conference on Data Engineering. IEEE Computer Society, 535–546.
[78]
John Thorpe, Yifan Qiao, Jonathan Eyolfson, Shen Teng, Guanzhou Hu, Zhihao Jia, Jinliang Wei, Keval Vora, Ravi Netravali, Miryung Kim, and Guoqing Harry Xu. 2021. Dorylus: Affordable, scalable, and accurate GNN training with distributed CPU servers and serverless threads. In 15th USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 495–514.
[79]
Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019. DyRep: Learning representations over dynamic graphs. In Proceedings of the International Conference on Learning Representations.
[80]
Manasi Vartak, Joana M. F. da Trindade, Samuel Madden, and Matei Zaharia. 2018. MISTIQUE: A system to store and query model intermediates for model diagnosis. In Proceedings of the 2018 International Conference on Management of Data. ACM, 1285–1300.
[81]
Manasi Vartak and Samuel Madden. 2018. MODELDB: Opportunities and challenges in managing machine learning models. IEEE Data Engineering Bulletin 41, 4 (2018), 16–25.
[82]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations.
[83]
Aladin Virmaux and Kevin Scaman. 2018. Lipschitz regularity of deep neural networks: Analysis and efficient estimation. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 3839–3848.
[84]
Cheng Wan, Youjie Li, Cameron R. Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin. 2022. PipeGCN: Efficient full-graph training of graph convolutional networks with pipelined feature communication. In Proceedings of the International Conference on Learning Representations.
[85]
Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang, Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, and Zhenyu Guo. 2021. APAN: Asynchronous propagation attention network for real-time temporal graph embedding. In Proceedings of the 2021 International Conference on Management of Data. ACM, 2628–2638.
[86]
Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. 2021. Inductive representation learning in temporal networks via causal anonymous walks. In Proceedings of the International Conference on Learning Representations.
[87]
Zhihao Wen and Yuan Fang. 2022. TREND: TempoRal event and node dynamics for graph representation learning. In Proceedings of the ACM Web Conference 2022. ACM, 1159–1169.
[88]
Doris Xin, Litian Ma, Jialin Liu, Stephen Macke, Shuchen Song, and Aditya G. Parameswaran. 2018. Accelerating human-in-the-loop machine learning: Challenges and opportunities. In Proceedings of the 2nd Workshop on Data Management for End-To-End Machine Learning, DEEM@SIGMOD. ACM, 9:1–9:4.
[89]
Doris Xin, Litian Ma, Jialin Liu, Stephen Macke, Shuchen Song, and Aditya G. Parameswaran. 2018. Helix: Accelerating human-in-the-loop machine learning. PVLDB 11, 12 (2018), 1958–1961.
[90]
Doris Xin, Stephen Macke, Litian Ma, Jialin Liu, Shuchen Song, and Aditya G. Parameswaran. 2018. Helix: Holistic optimization for accelerating iterative machine learning. PVLDB 12, 4 (2018), 446–460.
[91]
Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. 2020. Inductive representation learning on temporal graphs. In Proceedings of the International Conference on Learning Representations.
[92]
Guotong Xue, Ming Zhong, Jianxin Li, Jia Chen, Chengshuai Zhai, and Ruochen Kong. 2022. Dynamic network embedding survey. Neurocomputing 472 (2022), 212–223.
[93]
Han Yang, Kaili Ma, and James Cheng. 2021. Rethinking graph regularization for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 4573–4581.
[94]
Jiaxuan You, Tianyu Du, and Jure Leskovec. 2022. ROLAND: Graph learning framework for dynamic graphs. In Proceedings of the Knowledge Discovery and Data Mining. ACM, 2358–2366.
[95]
Haiyang Yu, Limei Wang, Bokun Wang, Meng Liu, Tianbao Yang, and Shuiwang Ji. 2022. GraphFM: Improving large-scale GNN training via feature momentum. In Proceedings of the International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 162). PMLR, 25684–25701.
[96]
Wenchao Yu, Wei Cheng, Charu C. Aggarwal, Kai Zhang, Haifeng Chen, and Wei Wang. 2018. NetWalk: A flexible deep embedding approach for anomaly detection in dynamic networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2672–2681.
[97]
Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor K. Prasanna. 2020. GraphSAINT: Graph sampling based inductive learning method. In Proceedings of the International Conference on Machine Learning.
[98]
Ce Zhang, Arun Kumar, and Christopher Ré. 2016. Materialization optimizations for feature selection workloads. ACM Transactions on Database Systems 41, 1 (2016), 2:1–2:32.
[99]
Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. 2020. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In Proceedings of the International Conference on Machine Learning. OpenReview.net.
[100]
Hongkuan Zhou, Da Zheng, Israt Nisa, Vassilis N. Ioannidis, Xiang Song, and George Karypis. 2022. TGL: A general framework for temporal GNN training on billion-scale graphs. PVLDB 15, 8 (2022), 1572–1580.
[101]
Kaixiong Zhou, Xiao Huang, Qingquan Song, Rui Chen, and Xia Hu. 2022. Auto-GNN: Neural architecture search of graph neural networks. Frontiers Big Data 5 (2022).
Index Terms
- A Caching-based Framework for Scalable Temporal Graph Neural Network Training
Recommendations
Orca: Scalable Temporal Graph Neural Network Training with Theoretical Guarantees
PACMMODRepresentation learning over dynamic graphs is critical for many real-world applications such as social network services and recommender systems. Temporal graph neural networks (T-GNNs) are powerful representation learning methods and have achieved ...
Counter-Based Cache Replacement and Bypassing Algorithms
Recent studies have shown that in highly associative caches, the performance gap between the Least Recently Used (LRU) and the theoretical optimal replacement algorithms is large, motivating the design of alternative replacement algorithms to improve ...
Compression-Based Program Characterization for Improving Cache Memory Performance
It is well known that compression and prediction are interrelated in that high compression implies good predictability, and vice versa. We use this correlation to find predictable properties of program behavior and apply them to appropriate cache ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 January 2025
Online AM: 25 November 2024
Accepted: 27 October 2024
Revised: 13 February 2024
Received: 05 May 2023
Published in TODS Volume 50, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key Research and Development Program of China
- Shanghai Municipal Science and Technology Major Project
- National Key Research and Development Program of China
- National Science Foundation of China (NSFC)
- Hong Kong RGC GRF Project
- RIF
- AOE Project
- Theme-based project TRS
- CRF Project
- Guangdong Province Science and Technology Plan Project
- Hong Kong ITC ITF
- Zhujiang scholar program
- Microsoft Research Asia Collaborative Research
- HKUST-Webank joint research lab and HKUST(GZ)-Chuanglin Graph Data Joint Lab
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 188Total Downloads
- Downloads (Last 12 months)188
- Downloads (Last 6 weeks)159
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in