research-article

Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks

Authors:

Xiaojun Chang, and

Chengqi ZhangAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

Pages 753 - 763

https://doi.org/10.1145/3394486.3403118

Published: 20 August 2020 Publication History

Abstract

Modeling multivariate time series has long been a subject that has attracted researchers from a diverse range of fields including economics, finance, and traffic. A basic assumption behind multivariate time series forecasting is that its variables depend on one another but, upon looking closely, it is fair to say that existing methods fail to fully exploit latent spatial dependencies between pairs of variables. In recent years, meanwhile, graph neural networks (GNNs) have shown high capability in handling relational dependencies. GNNs require well-defined graph structures for information propagation which means they cannot be applied directly for multivariate time series where the dependencies are not known in advance. In this paper, we propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module, into which external knowledge like variable attributes can be easily integrated. A novel mix-hop propagation layer and a dilated inception layer are further proposed to capture the spatial and temporal dependencies within the time series. The graph learning, graph convolution, and temporal convolution modules are jointly learned in an end-to-end framework. Experimental results show that our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets and achieves on-par performance with other approaches on two traffic datasets which provide extra structural information.

References

[1]

George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. 2015. Time series analysis: forecasting and control .John Wiley & Sons.

Digital Library

[2]

Fengwen Chen, Shirui Pan, Jing Jiang, Huan Huo, and Guodong Long. 2019 b. DAGCN: Dual Attention Graph Convolutional Networks. In Proc. of IJCNN.

[3]

Weiqi Chen, Ling Chen, Yu Xie, Wei Cao, Yusong Gao, and Xiaojie Feng. 2019 a. Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting. In Proc. of AAAI.

[4]

Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In Proc. of KDD.

Digital Library

[5]

Roger Frigola. 2015. Bayesian time series learning with Gaussian processes. Ph.D. Dissertation. University of Cambridge.

[6]

Roger Frigola-Alcalde. 2016. Bayesian time series learning with Gaussian processes. Ph.D. Dissertation. University of Cambridge.

[7]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In Proc. of ICML. 1263--1272.

[8]

Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proc. of AAAI, Vol. 33. 922--929.

Digital Library

[9]

Amol Kapoor, Aram Galstyan, Bryan Perozzi, Greg Ver Steeg, Hrayr Harutyunyan, Kristina Lerman, Nazanin Alipourfard, and Sami Abu-El-Haija. 2019. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. In Proc. of ICML.

[10]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proc. of ICLR.

[11]

Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then propagate: Graph neural networks meet personalized pagerank. In Proc. of ICLR.

[12]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In Proc. of SIGIR. ACM, 95--104.

Digital Library

[13]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proc. of ICLR.

[14]

Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv:1609.03499 (2016).

[15]

Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. 2019. Urban Traffic Prediction from Spatio-Temporal Data Using Deep Meta Learning. In KDD. ACM, 1720--1730.

[16]

Stephen Roberts, Michael Osborne, Mark Ebden, Steven Reece, Neale Gibson, and Suzanne Aigrain. 2013. Gaussian processes for time-series modelling. Philos. Trans. R. Soc. A, Vol. 371, 1984 (2013), 20110550.

[17]

Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. 2018. Structured sequence modeling with graph convolutional recurrent networks. In Proc. of NIPS. Springer, 362--373.

[18]

Lei Shi, Yifan Zhang, Jian Cheng, and Hanqing Lu. 2019. Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. In Proc. of CVPR. 12026--12035.

[19]

Shun-Yao Shih, Fan-Keng Sun, and Hung-yi Lee. 2019. Temporal pattern attention for multivariate time series forecasting. Machine Learning, Vol. 108, 8--9 (2019), 1421--1441.

Digital Library

[20]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proc. of CVPR. 1--9.

[21]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proc. of IJCAI.

[22]

Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proc. of AAAI. 3482--3489.

[23]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. In Proc. of IJCAI. 3634--3640.

[24]

Fisher Yu and Vladlen Koltun. 2016. Multi-scale context aggregation by dilated convolutions. In ICLR.

[25]

G Peter Zhang. 2003. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, Vol. 50 (2003), 159--175.

[26]

Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. GMAN: A Graph Multi-Attention Network for Traffic Prediction. In Proc. of AAAI.

Cited By

Wang DYang HZhou H(2024)Dynamic Spatial–Temporal Self-Attention Network for Traffic Flow PredictionFuture Internet10.3390/fi1606018916:6(189)Online publication date: 25-May-2024
https://doi.org/10.3390/fi16060189
Xie HFan XQi KWu DRen C(2024)Spatial–Temporal Fusion Gated Transformer Network (STFGTN) for Traffic Flow PredictionElectronics10.3390/electronics1308159413:8(1594)Online publication date: 22-Apr-2024
https://doi.org/10.3390/electronics13081594
Zong XChen ZYu FWei S(2024)Local-Global Spatial-Temporal Graph Convolutional Network for Traffic Flow ForecastingElectronics10.3390/electronics1303063613:3(636)Online publication date: 2-Feb-2024
https://doi.org/10.3390/electronics13030636
Show More Cited By

Index Terms

Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Dynamic personalized graph neural network with linear complexity for multivariate time series forecasting
Abstract
Spatio-temporal graph neural networks (GNN) that model inter-variable dependencies with learned graph structures for multivariate time series forecasting have received attention due to their superior performance. However, existing methods exhibit ...
Graphical abstract

Display Omitted
Highlights
- Spatio-temporal graph networks with linear complexity learn graph automatically.
- Dynamic personalized graph convolution enhances the influence of the nodes’ own.
- Experiments on six datasets demonstrate the effectiveness of the ...
Read More
Learning and integration of adaptive hybrid graph structures for multivariate time series forecasting
Abstract
Recent status-of-the-art methods for multivariate time series forecasting can be categorized into graph-based approach and global-local approach. The former approach uses graphs to represent the dependencies among variables and apply graph neural ...
Highlights
- Learn a global graph by sampling a PGM to capture the shared information and global dependencies among variables.
- Learn dynamic local graphs to capture the local randomness and correlations between pairs of variables.
- Devise a ...
Read More
Dynamic graph structure learning for multivariate time series forecasting
Highlights
- We formalize the problem of dependencies in multivariate time series data being a mixture of long- and short-term patterns.
Abstract
Multivariate time series forecasting is a challenging task because the dynamic spatio-temporal dependencies between variables are a combination of multiple unknown association patterns. Existing graph neural networks typically model ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '24

Sponsor:
sigkdd
sigkdd

The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 25 - 29, 2024

Barcelona , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

653
Total Citations
View Citations
8,872
Total Downloads

Downloads (Last 12 months)2,161
Downloads (Last 6 weeks)191

Other Metrics

View Author Metrics

Citations

Cited By

Wang DYang HZhou H(2024)Dynamic Spatial–Temporal Self-Attention Network for Traffic Flow PredictionFuture Internet10.3390/fi1606018916:6(189)Online publication date: 25-May-2024
https://doi.org/10.3390/fi16060189
Xie HFan XQi KWu DRen C(2024)Spatial–Temporal Fusion Gated Transformer Network (STFGTN) for Traffic Flow PredictionElectronics10.3390/electronics1308159413:8(1594)Online publication date: 22-Apr-2024
https://doi.org/10.3390/electronics13081594
Zong XChen ZYu FWei S(2024)Local-Global Spatial-Temporal Graph Convolutional Network for Traffic Flow ForecastingElectronics10.3390/electronics1303063613:3(636)Online publication date: 2-Feb-2024
https://doi.org/10.3390/electronics13030636
Ding ZHe ZHuang ZWang JYin H(2024)Traffic Flow Prediction Research Based on an Interactive Dynamic Spatial–Temporal Graph Convolutional Probabilistic Sparse Attention Mechanism (IDG-PSAtt)Atmosphere10.3390/atmos1504041315:4(413)Online publication date: 26-Mar-2024
https://doi.org/10.3390/atmos15040413
Huang HXie HXu ZLiu MXu YZhu T(2024)GenTrajRec: A Graph-Enhanced Trajectory Recovery Model Based on Signaling DataApplied Sciences10.3390/app1413593414:13(5934)Online publication date: 8-Jul-2024
https://doi.org/10.3390/app14135934
Gao LFan WJian W(2024)Dynamic Spatio-Temporal Adaptive Graph Convolutional Recurrent Networks for Vacant Parking Space PredictionApplied Sciences10.3390/app1413592714:13(5927)Online publication date: 7-Jul-2024
https://doi.org/10.3390/app14135927
Yang HWei SWang Y(2024)STFEformer: Spatial–Temporal Fusion Embedding Transformer for Traffic Flow PredictionApplied Sciences10.3390/app1410432514:10(4325)Online publication date: 20-May-2024
https://doi.org/10.3390/app14104325
Sun YChen CXu YXie SBlum RVenkitasubramaniam P(2024)On the generalization discrepancy of spatiotemporal dynamics-informed graph convolutional networksFrontiers in Mechanical Engineering10.3389/fmech.2024.139713110Online publication date: 12-Jul-2024
https://doi.org/10.3389/fmech.2024.1397131
Lu XTian FShen YZhang X(2024)Two-layer dynamic graph convolutional recurrent neural network for traffic flow predictionIntelligent Data Analysis10.3233/IDA-230174(1-17)Online publication date: 3-Jun-2024
https://doi.org/10.3233/IDA-230174
Jiang ZNing ZMiao HWang L(2024)STDNet: A Spatio-Temporal Decomposition Neural Network for Multivariate Time Series ForecastingTsinghua Science and Technology10.26599/TST.2023.901010529:4(1232-1247)Online publication date: Aug-2024
https://doi.org/10.26599/TST.2023.9010105
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents