research-article

Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning

Authors:

Lifang HeAuthors Info & Claims

Volume 578, Issue C

Pages 401 - 416

https://doi.org/10.1016/j.ins.2021.07.007

Published: 01 November 2021 Publication History

Abstract

Exploiting deep learning techniques for traffic flow prediction has become increasingly widespread. Most existing studies combine CNN or GCN with recurrent neural network to extract the spatio-temporal features in traffic networks. The traffic networks can be naturally modeled as graphs which are effective to capture the topology and spatial correlations among road links. The issue is that the traffic network is dynamic due to the continuous changing of the traffic environment. Compared with the static graph, the dynamic graph can better reflect the spatio-temporal features of the traffic network. However, in practical applications, due to the limited accuracy and timeliness of data, it is hard to generate graph structures through frequent statistical data. Therefore, it is necessary to design a method to overcome data defects in traffic flow prediction. In this paper, we propose a long-term traffic flow prediction method based on dynamic graphs. The traffic network is modeled by dynamic traffic flow probability graphs, and graph convolution is performed on the dynamic graphs to learn spatial features, which are then combined with LSTM units to learn temporal features. In particular, we further propose to use graph convolutional policy network based on reinforcement learning to generate dynamic graphs when the dynamic graphs are incomplete due to the data sparsity i sue. By testing our method on city-bike data in New York City, it demonstrates that our model can achieve stable and effective long-term predictions of traffic flow, and can reduce the impact of data defects on prediction results.

References

[1]

S. Wang, J. Cao, P. Yu, Deep learning for spatio-temporal data mining: A survey, IEEE Transactions on Knowledge and Data Engineering (2020).

[2]

Z. He, C.-Y. Chow, J.-D. Zhang, Stcnn: A spatio-temporal convolutional neural network for long-term traffic prediction, in: 2019 20th IEEE International Conference on Mobile Data Management (MDM), IEEE, 2019, pp. 226–233.

[3]

Y. Liu, H. Zheng, X. Feng, Z. Chen, Short-term traffic flow prediction with conv-lstm, in: 2017 9th International Conference on Wireless Communications and Signal Processing (WCSP), IEEE, 2017, pp. 1–6.

[4]

B. Du, H. Peng, S. Wang, M.Z.A. Bhuiyan, L. Wang, Q. Gong, L. Liu, J. Li, Deep irregular convolutional residual lstm for urban traffic passenger flows prediction, IEEE Transactions on Intelligent Transportation Systems (2019).

[5]

J. Li, H. Peng, L. Liu, G. Xiong, B. Du, H. Ma, L. Wang, M. Z. A. Bhuiyan, Graph cnns for urban traffic passenger flows prediction, in: 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), IEEE, 2018, pp. 29–36.

[6]

Z. Zhene, P. Hao, L. Lin, X. Guixi, B. Du, M. Z. A. Bhuiyan, Y. Long, D. Li, Deep convolutional mesh rnn for urban traffic passenger flows prediction, in: 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), IEEE, 2018, pp. 1305–1310.

[7]

Z. Cui, K. Henrickson, R. Ke, Y. Wang, Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting, IEEE Transactions on Intelligent Transportation Systems (2019).

[8]

L. Zhao, Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, H. Li, T-gcn: A temporal graph convolutional network for traffic prediction, IEEE Transactions on Intelligent Transportation Systems (2019).

[9]

B. Yu, H. Yin, Z. Zhu, Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting, in: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018, pp. 3634–3640.

[10]

H. Peng, H. Wang, B. Du, M.Z.A. Bhuiyan, H. Ma, J. Liu, L. Wang, Z. Yang, L. Du, S. Wang, et al., Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting, Information Sciences 521 (2020) 277–290.

Digital Library

[11]

L. Zhao, Y. Song, M. Deng, H. Li, Temporal graph convolutional network for urban traffic flow prediction method, arXiv preprint arXiv:1811.05320 (2018).

[12]

M. Uzunova, N. Linkov, R. Velichkova, K. Dimitrov, Bond graph based modeling approach with application for traffic flow, in: 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), IEEE, 2019, pp. 1705–1710.

[13]

X. Zhang, A. Haghani, S. Yang, Is dynamic traffic sensor network profitable for network-level real-time information prediction?, Transportation Research Part C: Emerging Technologies 102 (2019) 32–59.

[14]

C. Chen, K. Li, S. G. Teo, X. Zou, K. Wang, J. Wang, Z. Zeng, Gated residual recurrent graph neural networks for traffic prediction, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 485–492.

[15]

K. S. Oberoi, G. Del Mondo, Y. Dupuis, P. Vasseur, Modeling Road Traffic Takes Time, in: Proceedings of 10th International Conference on Geographic Information Science (GIScience 2018), Melbourne, Australia, 2018, pp. 52:1–52:7.

[16]

C. Chen, X. Liu, T. Qiu, A.K. Sangaiah, A short-term traffic prediction model in the vehicular cyber–physical systems, Future Generation Computer Systems (2017).

[17]

J. Jiang, N. Dellaert, T. Van Woensel, L. Wu, Modelling traffic flows and estimating road travel times in transportation network under dynamic disturbances, Transportation (2019) 1–30.

[18]

J. You, B. Liu, Z. Ying, V. Pande, J. Leskovec, Graph convolutional policy network for goal-directed molecular graph generation, in: Advances in Neural Information Processing Systems, 2018, pp. 6410–6421.

[19]

S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation 9 (1997) 1735–1780.

Digital Library

[20]

G. Atluri, A. Karpatne, V. Kumar, Spatio-temporal data mining: A survey of problems and methods, ACM Computing Surveys 51 (2017).

[21]

J. Zhang, Y. Zheng, J. Sun, D. Qi, Flow prediction in spatio-temporal networks based on multitask deep learning, IEEE Transactions on Knowledge and Data Engineering (2019).

[22]

J. Zhang, Y. Zheng, D. Qi, Deep spatio-temporal residual networks for citywide crowd flows prediction, in: Thirty-First AAAI Conference on Artificial Intelligence, 2017, p. 1655–1661.

[23]

G. Guo, T. Zhang, A residual spatio-temporal architecture for travel demand forecasting, Transportation Research Part C: Emerging Technologies 115 (2020).

[24]

Z. He, L. Zheng, P. Chen, W. Guan, Mapping to cells: a simple method to extract traffic dynamics from probe vehicle data, Computer-Aided Civil and Infrastructure Engineering 32 (2017) 252–267.

[25]

S. Guo, Y. Lin, S. Li, Z. Chen, H. Wan, Deep spatial-temporal 3d convolutional neural networks for traffic data forecasting, IEEE Transactions on Intelligent Transportation Systems 20 (2019) 3913–3926.

[26]

Y. Li, R. Yu, C. Shahabi, Y. Liu, Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, in: International Conference on Learning Representations, 2018.

[27]

S. Guo, Y. Lin, N. Feng, C. Song, H. Wan, Attention based spatial-temporal graph convolutional networks for traffic flow forecasting, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 922–929.

[28]

D. Chai, L. Wang, Q. Yang, Bike flow prediction with multi-graph convolutional networks, in: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2018, pp. 397–400.

[29]

T. Zhang, G. Guo, Graph attention lstm: A spatio-temperal approach for traffic flow forecasting, IEEE Intelligent Transportation Systems Magazine (2020).

[30]

Y. Wang, H. Yin, H. Chen, T. Wo, J. Xu, K. Zheng, Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 1227–1235.

Digital Library

[31]

C. Chen, K. Li, S.G. Teo, X. Zou, K. Li, Z. Zeng, Citywide traffic flow prediction based on multiple gated spatio-temporal convolutional neural networks, ACM Transactions on Knowledge Discovery from Data (TKDD) 14 (2020) 1–23.

[32]

L. Schultz, V. Sokolov, Deep reinforcement learning for dynamic urban transportation problems, 2018, arXiv preprint arXiv:1806.05310.

[33]

X.-Y. Liu, Z. Ding, S. Borst, A. Walid, Deep reinforcement learning for intelligent transportation systems, 2018, arXiv preprint arXiv:1812.00979.

[34]

R. Trivedi, M. Farajtabar, P. Biswal, H. Zha, Dyrep: Learning representations over dynamic graphs, in: International Conference on Learning Representations, 2019.

[35]

A. Sankar, Y. Wu, L. Gou, W. Zhang, H. Yang, Dynamic graph representation learning via self-attention networks, 2019.

[36]

P. Goyal, N. Kamra, X. He, Y. Liu, Dyngem: Deep embedding method for dynamic graphs, arXiv preprint arXiv:1805.11273 (2018).

[37]

A. Khan, E. Tolstaya, A. Ribeiro, V. Kumar, Graph policy gradients for large scale robot control, arXiv preprint arXiv:1907.03822 (2019).

[38]

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2, 2014, pp. 2672–2680.

[39]

T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: International Conference on Learning Representations, 2017.

[40]

L. Yu, W. Zhang, J. Wang, Y. Yu, Seqgan: Sequence generative adversarial nets with policy gradient, in: Thirty-First AAAI Conference on Artificial Intelligence, 2017.

[41]

S. Fujimoto, H. Hoof, D. Meger, Addressing function approximation error in actor-critic methods, in: International Conference on Machine Learning, PMLR, 2018, pp. 1587–1596.

[42]

B.M. Williams, L.A. Hoel, Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results, Journal of Transportation Engineering 129 (2003) 664–672.

[43]

A.J. Smola, B. Schölkopf, A tutorial on support vector regression, Statistics and Computing 14 (2004) 199–222.

[44]

Y. Tian, L. Pan, Predicting short-term traffic flow by long short-term memory recurrent neural network, in: 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity), IEEE, 2015, pp. 153–158.

[45]

Y. Li, R. Yu, C. Shahabi, Y. Liu, Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, in: International Conference on Learning Representations, 2018.

[46]

F. Wu, T. Zhang, A. H. d. Souza Jr, C. Fifty, T. Yu, K. Q. Weinberger, Simplifying graph convolutional networks, Proceedings of Machine Learning Research (2019).

[47]

X. Geng, Y. Li, L. Wang, L. Zhang, Q. Yang, J. Ye, Y. Liu, Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 3656–3663.

[48]

P. He, G. Jiang, S.-K. Lam, Y. Sun, Learning heterogeneous traffic patterns for travel time prediction of bus journeys, Information Sciences 512 (2020) 1394–1406.

[49]

W. Cheng, Y. Shen, Y. Zhu, L. Huang, A neural attention model for urban air quality inference: Learning the weights of monitoring stations, in: Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

[50]

Y. Zhou, J. Li, H. Chen, Y. Wu, J. Wu, L. Chen, A spatiotemporal hierarchical attention mechanism-based model for multi-step station-level crowd flow prediction, Information Sciences 544 (2021) 308–324.

Cited By

Tian YLiu LFeng JPei QChen CDu JWu C(2024)Toward Robust and Generalizable Federated Graph Neural Networks for Decentralized Spatial-Temporal Data ModelingIEEE Transactions on Network and Service Management10.1109/TNSM.2024.338674021:3(2637-2650)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3386740
Jin GLiang YFang YShao ZHuang JZhang JZheng Y(2024)Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333382436:10(5388-5408)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3333824
Mallick TMacfarlane JBalaprakash P(2024)Uncertainty Quantification for Traffic Forecasting Using Deep-Ensemble-Based Spatiotemporal Graph Neural NetworksIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.338109925:8(9141-9152)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3381099
Show More Cited By

Index Terms

Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Interactive dynamic diffusion graph convolutional network for traffic flow prediction
Abstract
Capturing the temporal and spatial features, and the spatiotemporal correlations in traffic networks is the essential task for accurate traffic flow prediction. Although most existing models have explored the temporal features deeply, the ...
An Urban Traffic Knowledge Graph-Driven Spatial-Temporal Graph Convolutional Network for Traffic Flow Prediction
IJCKG '22: Proceedings of the 11th International Joint Conference on Knowledge Graphs

Traffic flow prediction is a critical issue for researchers and practitioners in the field of transportation. Due to the high nonlinearity and complexity of traffic data, deep learning approaches have attracted much interest in recent years. However, ...
Spatial dynamic graph convolutional network for traffic flow forecasting
Abstract
The complex traffic network spatial correlation and the characteristic of high nonlinear and dynamic traffic conditions in the time are the challenges to accurate traffic flow forecasting. Existing spatiotemporal models attempt to utilize the ...

Comments

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 578, Issue C

Nov 2021

950 pages

ISSN:0020-0255

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 November 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tian YLiu LFeng JPei QChen CDu JWu C(2024)Toward Robust and Generalizable Federated Graph Neural Networks for Decentralized Spatial-Temporal Data ModelingIEEE Transactions on Network and Service Management10.1109/TNSM.2024.338674021:3(2637-2650)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3386740
Jin GLiang YFang YShao ZHuang JZhang JZheng Y(2024)Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333382436:10(5388-5408)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TKDE.2023.3333824
Mallick TMacfarlane JBalaprakash P(2024)Uncertainty Quantification for Traffic Forecasting Using Deep-Ensemble-Based Spatiotemporal Graph Neural NetworksIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.338109925:8(9141-9152)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3381099
Gao CLiu HHuang JWang ZLi XLi X(2024)Regularized Spatial–Temporal Graph Convolutional Networks for Metro Passenger Flow PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.336517925:9(11241-11255)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3365179
Wang BLi HWang WWang MJin YXu Y(2024)PG² Net: Personalized and Group Preferences Guided Network for Next Place PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335529225:8(8655-8670)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3355292
Hu HHu QTan GZhang YLin Z(2024)A Multi-Layer Model Based on Transformer and Deep Learning for Traffic Flow PredictionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.331139725:1(443-451)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TITS.2023.3311397
Liu LSi HKarimi H(2024)Intelligent emergency traffic signal control system with pedestrian accessInformation Sciences: an International Journal10.1016/j.ins.2024.120805679:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120805
Bi JDornaika F(2024)Sample-weighted fused graph-based semi-supervised learning on multi-view dataInformation Fusion10.1016/j.inffus.2023.102175104:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.inffus.2023.102175
Chen JZheng LHu YWang WZhang HHu X(2024)Traffic flow matrix-based graph neural network with attention mechanism for traffic flow predictionInformation Fusion10.1016/j.inffus.2023.102146104:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.inffus.2023.102146
Wang CZuo KZhang SLiu CPeng HLi WShen ZHu PWang RJie B(2024)TADGCNExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125134258:COnline publication date: 15-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.125134
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents