Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3459637.3482014acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models

Published: 30 October 2021 Publication History

Abstract

We present PyTorch Geometric Temporal, a deep learning framework combining state-of-the-art machine learning algorithms for neural spatiotemporal signal processing. The main goal of the library is to make temporal geometric deep learning available for researchers and machine learning practitioners in a unified easy-to-use framework. PyTorch Geometric Temporal was created with foundations on existing libraries in the PyTorch eco-system, streamlined neural network layer definitions, temporal snapshot generators for batching, and integrated benchmark datasets. These features are illustrated with a tutorial-like case study. Experiments demonstrate the predictive performance of the models implemented in the library on real-world problems such as epidemiological forecasting, ride-hail demand prediction, and web traffic management. Our sensitivity analysis of runtime shows that the framework can potentially operate on web-scale datasets with rich temporal features and spatial structure.

References

[1]
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A System for Large-Scale Machine Learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265--283.
[2]
Sean Anderson, Eric Ward, Lewis Barnett, and Philippina English. 2018. sdmTMB: Spatial and spatiotemporal GLMMs with TMB. https://github.com/pbs-assess/sdmTMB.
[3]
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations, ICLR 2015.
[4]
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting. Advances in Neural Information Processing Systems, Vol. 33 (2020).
[5]
Ferenc Béres, Domokos M. Kelen, Róbert Pálovics, and András A. Benczúr. 2019. Node Embeddings in Dynamic Graphs. Applied Network Science, Vol. 4, 64 (2019), 25.
[6]
Ferenc Béres, Róbert Pálovics, Anna Oláh, and András A. Benczúr. 2018. Temporal Walk Based Centrality Metric for Graph Streams. Applied Network Science, Vol. 3, 32 (2018), 26.
[7]
Aleksandar Bojchevski, Johannes Klicpera, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, and Stephan Günnemann. 2020. Scaling Graph Neural Networks with Approximate Pagerank. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2464--2473.
[8]
Georg Brandl. 2010. Sphinx Documentation. URL http://sphinx-doc.org/sphinx. pdf (2010).
[9]
Yukuo Cen, Zhenyu Hou, Yan Wang, Qibin Chen, Yizhen Luo, Xingcheng Yao, Aohan Zeng, Shiguang Guo, Peng Zhang, Guohao Dai, et al. 2021. CogDL: An Extensive Toolkit for Deep Learning on Graphs. arXiv preprint arXiv:2103.00959 (2021).
[10]
Jinyin Chen, Xuanheng Xu, Yangyang Wu, and Haibin Zheng. 2018. GC-LSTM: Graph Convolution Embedded LSTM for Dynamic Link Prediction. arXiv preprint arXiv:1812.04206 (2018).
[11]
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. arXiv preprint arXiv:1512.01274 (2015).
[12]
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations Using RNN Encoder--Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 1724--1734.
[13]
CSIRO's Data61. 2018. StellarGraph Machine Learning Library. https://github.com/stellargraph/stellargraph.
[14]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Advances in Neural Information Processing Systems. 3844--3852.
[15]
Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
[16]
Roy Frostig, Matthew James Johnson, and Chris Leary. 2018. Compiling Machine Learning Programs via High-Level Tracing. Systems for Machine Learning (2018).
[17]
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In International Conference on Machine Learning. PMLR, 1263--1272.
[18]
Jonathan Godwin, Thomas Keck, Peter Battaglia, Victor Bapst, Thomas Kipf, Yujia Li, Kimberly Stachenfeld, Petar Velivcković, and Alvaro Sanchez-Gonzalez. 2020. Jraph: A Library for Graph Neural Networks in Jax. http://github.com/deepmind/jraph
[19]
Palash Goyal, Sujit Rokka Chhetri, Ninareh Mehrabi, Emilio Ferrara, and Arquimedes Canedo. 2018. DynamicGEM: A Library for Dynamic Graph Embedding Methods. arXiv preprint arXiv:1811.10734 (2018).
[20]
Palash Goyal and Emilio Ferrara. [n.d.]. GEM: A Python Package for Graph Embedding Methods. Journal of Open Source Software, Vol. 3, 29 ( [n.,d.]), 876.
[21]
Daniele Grattarola and Cesare Alippi. 2020. Graph Neural Networks in TensorFlow and Keras with Spektral. arXiv preprint arXiv:2006.12138 (2020).
[22]
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 922--929.
[23]
Matthew Hanson. 2019. The Open-Source Software Ecosystem for Leveraging Public Datasets in Spatio-Temporal Asset Catalogs (STAC). In AGU Fall Meeting Abstracts, Vol. 2019. IN23B--07.
[24]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.
[25]
Petter Holme. 2015. Modern Temporal Network Theory: A Colloquium. The European Physical Journal B, Vol. 88, 9 (2015), 1--30.
[26]
Petter Holme and Jari Saram"aki. 2012. Temporal Networks. Physics reports, Vol. 519, 3 (2012), 97--125.
[27]
Jun Hu, Shengsheng Qian, Quan Fang, Youze Wang, Quan Zhao, Huaiwen Zhang, and Changsheng Xu. 2021. Efficient Graph Deep Learning in TensorFlow with TF Geometric. arXiv preprint arXiv:2101.11552 (2021).
[28]
James Bradbury and Roy Frostig and Peter Hawkins and Matthew James Johnson and Chris Leary and Dougal Maclaurin and George Necula and Adam Paszke and Jake VanderPlas and Skye Wanderman-Milne and Qiao Zhang. 2018. JAX: Composable Transformations of Python+NumPy Programs. http://github.com/google/jax
[29]
Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations.
[30]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations (ICLR).
[31]
Jia Li, Zhichao Han, Hong Cheng, Jiao Su, Pengyun Wang, Jianfeng Zhang, and Lujia Pan. 2019. Predicting Path Failure in Time-Evolving Graphs. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1279--1289.
[32]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations.
[33]
Yujia Li, Richard Zemel, Marc Brockschmidt, and Daniel Tarlow. 2016. Gated Graph Sequence Neural Networks. In International Conference on Learning Representations (ICLR).
[34]
Meng Liu, Youzhi Luo, Limei Wang, Yaochen Xie, Hao Yuan, Shurui Gui, Zhao Xu, Haiyang Yu, Jingtun Zhang, Yi Liu, Keqiang Yan, Bora Oztekin, Haoran Liu, Xuan Zhang, Cong Fu, and Shuiwang Ji. 2021. DIG: A Turnkey Library for Diving into Graph Deep Learning Research. arXiv preprint arXiv:2103.12608 (2021).
[35]
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1412--1421.
[36]
Harold B Maynard, G J Stegemerten, and John L Schwab. 1948. Methods-Time Measurement. McGraw-Hill.
[37]
Vinod Nair and Geoffrey E Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning. 807--814.
[38]
George Panagopoulos, Giannis Nikolentzos, and Michalis Vazirgiannis. 2021. Transfer Graph Neural Networks for Pandemic Forecasting. In Proceedings of the 35th AAAI Conference on Artificial Intelligence.
[39]
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B Schardl, and Charles E Leiserson. 2020. EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs. In AAAI. 5363--5370.
[40]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019 a. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems. 8024--8035.
[41]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019 b. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, Vol. 32 (2019), 8026--8037.
[42]
Edzer Pebesma. 2017. staRs: Spatiotemporal Arrays: Raster and Vector Datacubes. https://github.com/r-spatial/stars.
[43]
Sergio J Rey and Luc Anselin. 2010. PySAL: A Python Library of Spatial Analytical Methods. In Handbook of Applied Spatial Analysis. Springer, 175--193.
[44]
Emanuele Rob. 2020. PySTAC: Python library for working with any SpatioTemporal Asset Catalog (STAC). https://github.com/stac-utils/pystac. GitHub repository.
[45]
Benedek Rozemberczki, Peter Englert, Amol Kapoor, Martin Blais, and Bryan Perozzi. 2020 a. Pathfinder Discovery Networks for Neural Message Passing. arXiv preprint arXiv:2010.12878 (2020).
[46]
Benedek Rozemberczki, Oliver Kiss, and Rik Sarkar. 2020 b. Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 3125--3132.
[47]
Benedek Rozemberczki, Paul Scherer, Oliver Kiss, Rik Sarkar, and Tamas Ferenci. 2021. Chickenpox Cases in Hungary: a Benchmark Dataset for Spatiotemporal Signal Processing with Graph Neural Networks .arxiv: 2102.08100 [cs.LG]
[48]
Jason Sanders and Edward Kandrot. 2010. CUDA by Example: An Introduction to General-Purpose GPU Programming. Addison-Wesley Professional.
[49]
Paul Scherer and Pietro Lio. 2020. Learning Distributed Representations of Graphs with Geo2DR. In ICML Workshop on Graph Representation Learning and Beyond.
[50]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In European Semantic Web Conference. Springer, 593--607.
[51]
Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. 2018. Structured Sequence Modeling with Graph Convolutional Recurrent Networks. In International Conference on Neural Information Processing. Springer, 362--373.
[52]
Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. 2019. Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12026--12035.
[53]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. The Journal of Machine Learning Research, Vol. 15, 1 (2014), 1929--1958.
[54]
Aynaz Taheri and Tanya Berger-Wolf. 2019. Predictive Temporal Embedding of Dynamic Graphs. In Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. 57--64.
[55]
Aynaz Taheri, Kevin Gimpel, and Tanya Berger-Wolf. 2019. Learning to Represent the Evolution of Dynamic Graphs with Recurrent Models. In Companion Proceedings of The 2019 World Wide Web Conference (WWW '19). 301--307.
[56]
Paul Taylor, Christopher Harris, Thompson Comer, and Mark Harris. 2019. CUDA-Accelerated GIS and Spatiotemporal Algorithms. https://github.com/rapidsai/cuspatial.
[57]
Cunchao Tu, Yuan Yao, Zhengyan Zhang, Ganqu Cui, Hao Wang, Changxin Tian, Jie Zhou, and Cheng Yang. 2018. OpenNE: An Open Source Toolkit for Network Embedding. https://github.com/thunlp/OpenNE.
[58]
Stefan Van Der Walt, S. Chris Colbert, and Gael Varoquaux. 2011. The NumPy Array: a Structure for Efficient Numerical Computation. Computing in science & engineering, Vol. 13, 2 (2011), 22--30.
[59]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6000--6010.
[60]
Michael A Whitby, Rich Fecher, and Chris Bennight. 2017. GeoWave: Utilizing Distributed Key-Value Stores for Multidimensional Data. In International Symposium on Spatial and Temporal Databases. Springer, 105--122.
[61]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 753--763.
[62]
Hongxia Yang. 2019. AliGraph: A Comprehensive Graph Neural Network Platform. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3165--3166.
[63]
Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Convolutional Networks: a Deep Learning Framework for Traffic Forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3634--3640.
[64]
Le Yu, Leilei Sun, Bowen Du, Chuanren Liu, Hui Xiong, and Weifeng Lv. 2020. Predicting Temporal Sets with Deep Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1083--1091.
[65]
Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, and Matthias Grundmann. 2020. MediaPipe Hands: On-device Real-time Hand Tracking .arxiv: 2006.10214 [cs.CV]
[66]
Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. 2019. T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction. IEEE Transactions on Intelligent Transportation Systems, Vol. 21, 9 (2019), 3848--3858.
[67]
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020a. GMAN: A Graph Multi-Attention Network for Traffic Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1234--1241.
[68]
Da Zheng, Minjie Wang, Quan Gan, Zheng Zhang, and George Karypis. 202 b. Learning Graph Neural Networks with Deep Graph Library. In Companion Proceedings of the Web Conference 2020 (WWW '20). 305--306.
[69]
Jiawei Zhu, Yujiao Song, Ling Zhao, and Haifeng Li. 2020. A3T-GCN: Attention Temporal Graph Convolutional Network for Traffic Forecasting. arXiv preprint arXiv:2006.11583 (2020).
[70]
Esteban Zimányi, Mahmoud Sakr, and Arthur Lesuisse. 2020. MobilityDB: A Mobility Database Based on PostgreSQL and PostGIS. ACM Transactions on Database Systems (TODS), Vol. 45, 4 (2020), 1--42.

Cited By

View all
  • (2024)Towards Generalizability of Multi-Agent Reinforcement Learning in Graphs with Recurrent Message PassingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663055(1919-1927)Online publication date: 6-May-2024
  • (2024)Geospatial Machine Learning and the Power of Python ProgrammingEthics, Machine Learning, and Python in Geospatial Analysis10.4018/979-8-3693-6381-2.ch010(223-253)Online publication date: 10-May-2024
  • (2024)Graph-Based Traffic Forecasting with the Dynamics of Road Symmetry and Capacity PerformanceSymmetry10.3390/sym1607093516:7(935)Online publication date: 22-Jul-2024
  • Show More Cited By

Index Terms

  1. PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management
    October 2021
    4966 pages
    ISBN:9781450384469
    DOI:10.1145/3459637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 October 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Best Paper

    Author Tags

    1. deep learning
    2. graph neural networks
    3. machine learning
    4. time series data

    Qualifiers

    • Research-article

    Conference

    CIKM '21
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

    Upcoming Conference

    CIKM '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)344
    • Downloads (Last 6 weeks)27
    Reflects downloads up to 23 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Towards Generalizability of Multi-Agent Reinforcement Learning in Graphs with Recurrent Message PassingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663055(1919-1927)Online publication date: 6-May-2024
    • (2024)Geospatial Machine Learning and the Power of Python ProgrammingEthics, Machine Learning, and Python in Geospatial Analysis10.4018/979-8-3693-6381-2.ch010(223-253)Online publication date: 10-May-2024
    • (2024)Graph-Based Traffic Forecasting with the Dynamics of Road Symmetry and Capacity PerformanceSymmetry10.3390/sym1607093516:7(935)Online publication date: 22-Jul-2024
    • (2024)Data-Driven Wildfire Spread Modeling of European Wildfires Using a Spatiotemporal Graph Neural NetworkFire10.3390/fire70602077:6(207)Online publication date: 19-Jun-2024
    • (2024)Hardware Acceleration of Inference on Dynamic GNNsProceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design10.1145/3665314.3670832(1-6)Online publication date: 5-Aug-2024
    • (2024)TG-SPRED: Temporal Graph for Sensorial Data PREDictionACM Transactions on Sensor Networks10.1145/364989220:3(1-20)Online publication date: 13-Apr-2024
    • (2024)GraphStorm: All-in-one Graph Machine Learning Framework for Industry ApplicationsProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671603(6356-6367)Online publication date: 25-Aug-2024
    • (2024)Parallel-friendly Spatio-Temporal Graph Learning for Photovoltaic Degradation Analysis at ScaleProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680026(4470-4478)Online publication date: 21-Oct-2024
    • (2024)BehaviorNet: A Fine-grained Behavior-aware Network for Dynamic Link PredictionACM Transactions on the Web10.1145/358051418:2(1-26)Online publication date: 8-Jan-2024
    • (2024)Efficiency Improvisation of Large-Scale Knowledge Systems in Feature Determination using Proposed HVGAN ArchitectureJournal of Information & Knowledge Management10.1142/S021964922450006023:02Online publication date: 23-Jan-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media