Cited By
View all- Peng HZhang JHuang XHao ZLi AYu ZYu P(undefined)Unsupervised Social Bot Detection via Structural Information TheoryACM Transactions on Information Systems10.1145/3660522
MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization
Authors: 
Dongcheng Zou, Senzhang Wang, Xuefeng Li, Hao Peng, Yuandong Wang, Chunyang Liu, Kehua Sheng, Bo ZhangAuthors Info & Claims
Dongcheng Zou, Senzhang Wang, Xuefeng Li, Hao Peng, Yuandong Wang, Chunyang Liu, Kehua Sheng, Bo ZhangAuthors Info & ClaimsPages 1032 - 1041
Abstract
Traffic forecasting is a complex multivariate time-series regression task of paramount importance for traffic management and planning. However, existing approaches often struggle to model complex multi-range dependencies using local spatiotemporal features and road network hierarchical knowledge. To address this, we propose MultiSPANS. First, considering that an individual recording point cannot reflect critical spatiotemporal local patterns, we design multi-filter convolution modules for generating informative ST-token embeddings to facilitate attention computation. Then, based on ST-token and spatial-temporal position encoding, we employ the Transformers to capture long-range temporal and spatial dependencies. Furthermore, we introduce structural entropy theory to optimize the spatial attention mechanism. Specifically, The structural entropy minimization algorithm is used to generate optimal road network hierarchies, i.e., encoding trees. Based on this, we propose a relative structural entropy-based position encoding and a multi-head attention masking scheme based on multi-layer encoding trees. Extensive experiments demonstrate the superiority of the presented framework over several state-of-the-art methods in real-world traffic datasets, and the longer historical windows are effectively utilized. The code is available at https://github.com/SELGroup/MultiSPANS.
Supplementary Material
The presentation video of 'MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization'.
- Download
- 39.45 MB
References
[1]
Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learning. PMLR, 21--29.
[2]
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting. Advances in Neural Information Processing Systems 33 (2020), 17804--17815.
[3]
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020b. Simple and deep graph convolutional networks. In International conference on machine learning. PMLR, 1725--1735.
[4]
Weiqi Chen, Ling Chen, Yu Xie, Wei Cao, Yusong Gao, and Xiaojie Feng. 2020a. Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 34, 04 (April 2020), 3529--3536. https://doi.org/10.1609/aaai.v34i04.5758
[5]
Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. 2019. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. IEEE Transactions on Intelligent Transportation Systems 21, 11 (2019), 4883--4894.
[6]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations.
[7]
Harris Drucker, Christopher J. C. Burges, Linda Kaufman, Alex Smola, and Vladimir Vapnik. 1996. Support Vector Regression Machines. In Advances in Neural Information Processing Systems, Vol. 9. MIT Press.
[8]
Vijay Prakash Dwivedi and Xavier Bresson. 2021. A Generalization of Transformer Networks to Graphs. https://doi.org/10.48550/arXiv.2012.09699 arxiv: 2012.09699 [cs]
[9]
Kan Guo, Yongli Hu, Yanfeng Sun, Sean Qian, Junbin Gao, and Baocai Yin. 2021. Hierarchical Graph Convolution Network for Traffic Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 35, 1 (May 2021), 151--159. https://doi.org/10.1609/aaai.v35i1.16088
[10]
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019a. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (July 2019), 922--929. https://doi.org/10.1609/aaai.v33i01.3301922
[11]
Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019b. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (July 2019), 922--929. https://doi.org/10.1609/aaai.v33i01.3301922
[12]
Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. 2022. Learning Dynamics and Heterogeneity of Spatial-Temporal Graph Data for Traffic Forecasting. IEEE Transactions on Knowledge and Data Engineering 34, 11 (Nov. 2022), 5415--5428. https://doi.org/10.1109/TKDE.2021.3056502
[13]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (Nov. 1997), 1735--1780. https://doi.org/10.1162/neco.1997.9.8.1735
[14]
Chao Huang, Chuxu Zhang, Peng Dai, and Liefeng Bo. 2019. Deep dynamic fusion network for traffic accident forecasting. In Proceedings of the 28th ACM international conference on information and knowledge management. 2673--2681.
[15]
Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. 2023. PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction.
[16]
Angsheng Li, Qifu Hu, Jun Liu, and Yicheng Pan. 2016a. Resistance and security index of networks: structural information perspective of network security. Scientific reports 6, 1 (2016), 26810.
[17]
Angsheng Li and Yicheng Pan. 2016. Structural information and dynamical complexity of networks. IEEE Transactions on Information Theory 62, 6 (2016), 3290--3339.
[18]
Angsheng Li, Xianchen Yin, and Yicheng Pan. 2016b. Three-dimensional gene map of cancer cell types: Structural entropy minimisation principle for defining tumour subtypes. Scientific reports 6, 1 (2016), 1--26.
[19]
Angsheng Li, Xianchen Yin, Bingxiang Xu, Danyang Wang, Jimin Han, Yi Wei, Yun Deng, Ying Xiong, and Zhihua Zhang. 2018. Decoding topologically associating domains with ultra-low resolution Hi-C data by graph structural entropy. Nature communications 9, 1 (2018), 1--12.
[20]
Angsheng Li, Xiaohui Zhang, and Yicheng Pan. 2017. Resistance maximization principle for defending networks against virus attack. Physica A: Statistical Mechanics and its Applications 466 (2017), 211--223.
[21]
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2022. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations.
[22]
Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. 2023. MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing.
[23]
Defu Lian, Yongji Wu, Yong Ge, Xing Xie, and Enhong Chen. 2020. Geography-aware sequential location recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2009--2019.
[24]
Yiwei Liu, Jiamou Liu, Zijian Zhang, Liehuang Zhu, and Angsheng Li. 2019. REM: From structural entropy to community structure deception. Advances in Neural Information Processing Systems 32 (2019).
[25]
Ilya Loshchilov and Frank Hutter. 2023. Fixing Weight Decay Regularization in Adam. (May 2023).
[26]
Bin Lu, Xiaoying Gan, Haiming Jin, Luoyi Fu, and Haisong Zhang. 2020. Spatiotemporal Adaptive Gated Graph Convolution Network for Urban Traffic Flow Forecasting. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM '20). Association for Computing Machinery, New York, NY, USA, 1025--1034. https://doi.org/10.1145/3340531.3411894
[27]
Zheng Lu, Chen Zhou, Jing Wu, Hao Jiang, and Songyue Cui. 2016. Integrating Granger Causality and Vector Auto-Regression for Traffic Prediction of Large-Scale WLANs. KSII Transactions on Internet and Information Systems 10, 1 (Jan. 2016), 136--151.
[28]
Yingtao Luo, Qiang Liu, and Zhaocheng Liu. 2021. Stan: Spatio-temporal attention network for next location recommendation. In Proceedings of the Web Conference 2021. 2177--2185.
[29]
Yisheng Lv, Yanjie Duan, Wenwen Kang, Zhengxi Li, and Fei-Yue Wang. 2015. Traffic Flow Prediction With Big Data: A Deep Learning Approach. IEEE Transactions on Intelligent Transportation Systems 16, 2 (April 2015), 865--873. https://doi.org/10.1109/TITS.2014.2345663
[30]
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. arXiv preprint arXiv:2211.14730 (2023).
[31]
Yamila M. Omar and Peter Plapper. 2020. A Survey of Information Entropy Metrics for Complex Networks. Entropy, Vol. 22, 12 (Dec. 2020), 1417. https://doi.org/10.3390/e22121417
[32]
Cheonbok Park, Chunggi Lee, Hyojin Bahng, Yunwon Tae, Seungmin Jin, Kihwan Kim, Sungahn Ko, and Jaegul Choo. 2020. ST-GRAT: A Novel Spatio-Temporal Graph Attention Networks for Accurately Forecasting Dynamically Changing Road Speed. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM '20). Association for Computing Machinery, New York, NY, USA, 1215--1224. https://doi.org/10.1145/3340531.3411940
[33]
Martin Rosvall and Carl T Bergstrom. 2008. Maps of random walks on complex networks reveal community structure. Proceedings of the national academy of sciences 105, 4 (2008), 1118--1123.
[34]
Chao Shang, Jie Chen, and Jinbo Bi. 2022. Discrete Graph Structure Learning for Forecasting Multiple Time Series. In International Conference on Learning Representations.
[35]
Claude Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal 27, 3 (1948), 379--423.
[36]
Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. 2022. Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM '22). Association for Computing Machinery, New York, NY, USA, 4454--4458. https://doi.org/10.1145/3511808.3557702
[37]
Ke Sun, Tieyun Qian, Tong Chen, Yile Liang, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2020. Where to go next: Modeling long-and short-term user preferences for point-of-interest recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 214--221.
[38]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc.
[39]
Beibei Wang, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2021b. GSNet: learning spatial-temporal correlations from geographical and semantic aspects for traffic accident risk forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4402--4409.
[40]
Jingyuan Wang, Jiawei Jiang, Wenjun Jiang, Chao Li, and Wayne Xin Zhao. 2021a. LibCity: An Open Library for Traffic Prediction. In Proceedings of the 29th International Conference on Advances in Geographic Information Systems (SIGSPATIAL '21). Association for Computing Machinery, New York, NY, USA, 145--148. https://doi.org/10.1145/3474717.3483923
[41]
Senzhang Wang, Jiannong Cao, and S Yu Philip. 2020. Deep learning for spatio-temporal data mining: A survey. IEEE transactions on knowledge and data engineering 34, 8 (2020), 3681--3700.
[42]
Yue Wang, Mingsheng Liu, Yongjian Huang, Haifeng Zhou, Xianhui Wang, Senzhang Wang, and Haohua Du. 2022. Knowledge-based and data-driven underground pressure forecasting based on graph structure learning. International Journal of Machine Learning and Cybernetics (2022), 1--16.
[43]
Yuandong Wang, Hongzhi Yin, Hongxu Chen, Tianyu Wo, Jie Xu, and Kai Zheng. 2019. Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 1227--1235.
[44]
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34 (2021), 22419--22430.
[45]
Junran Wu, Xueyuan Chen, Ke Xu, and Shangzhe Li. 2022. Structural entropy guided graph hierarchical pooling. In Proceedings of the International Conference on Machine Learning. PMLR, 24017--24030.
[46]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the Dots : Multivariate Time Series Forecasting with Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD '20). Association for Computing Machinery, New York, NY, USA, 753--763. https://doi.org/10.1145/3394486.3403118
[47]
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence (AAAI), 1907--1913. https://doi.org/10.24963/ijcai.2019/264
[48]
Jiangnan Xia, Senzhang Wang, Xiang Wang, Min Xia, Kun Xie, and Jiannong Cao. 2022. Multi-view Bayesian spatio-temporal graph neural networks for reliable traffic flow prediction. International Journal of Machine Learning and Cybernetics (2022), 1--14.
[49]
Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, and Hongkai Xiong. 2021. Spatial-Temporal Transformer Networks for Traffic Flow Forecasting.
[50]
Zhenyu Yang, Ge Zhang, Jia Wu, Jian Yang, Quan Z Sheng, Hao Peng, Angsheng Li, Shan Xue, and Jianlin Su. 2023. Minimum entropy principle guided graph neural networks. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 114--122.
[51]
Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, and Zhenhui Li. 2019. Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (July 2019), 5668--5675. https://doi.org/10.1609/aaai.v33i01.33015668
[52]
Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, and Hui Xiong. 2021. Coupled layer-wise graph convolution for transportation demand prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4617--4625.
[53]
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems 34 (2021), 28877--28888.
[54]
Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, Stockholm, Sweden, 3634--3640. https://doi.org/10.24963/ijcai.2018/505
[55]
Zhuoning Yuan, Xun Zhou, and Tianbao Yang. 2018. Hetero-convlstm: A deep learning approach to traffic accident prediction on heterogeneous spatio-temporal data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 984--992.
[56]
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2022. Are Transformers Effective for Time Series Forecasting?
[57]
Guangjie Zeng, Hao Peng, Angsheng Li, Zhiwei Liu, Chunyang Liu, Philip S Yu, and Lifang He. 2023 c. Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphs. arXiv preprint arXiv:2309.01899 (2023).
[58]
Xianghua Zeng, Hao Peng, and Angsheng Li. 2023. Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principles. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 11772--11780.
[59]
Xianghua Zeng, Hao Peng, Angsheng Li, Chunyang Liu, Lifang He, and Philip S Yu. 2023. Hierarchical State Abstraction based on Structural Information Principles. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence.
[60]
Qi Zhang, Jianlong Chang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2020. Spatio-Temporal Graph Structure Learning for Traffic Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 34, 01 (April 2020), 1177--1185. https://doi.org/10.1609/aaai.v34i01.5470
[61]
Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. 2020. T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction. IEEE Transactions on Intelligent Transportation Systems 21, 9 (Sept. 2020), 3848--3858. https://doi.org/10.1109/TITS.2019.2935152
[62]
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. GMAN: A Graph Multi-Attention Network for Traffic Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence (2020-04-03), Vol. 34. 1234--1241. Issue 01. https://doi.org/10.1609/aaai.v34i01.5477
[63]
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 11106--11115.
[64]
Dongcheng Zou, Hao Peng, Xiang Huang, Renyu Yang, Jianxin Li, Jia Wu, Chunyang Liu, and Philip S Yu. 2023. SE-GSL: A General and Effective Graph Structure Learning Framework through Structural Entropy Optimization. In Proceedings of the ACM Web Conference 2023. 499--510. io
Index Terms
- MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization
Recommendations
Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementWith the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous ...
An Engineering Approach to Prediction of Network Traffic Based on Time-Series Model
JCAI '09: Proceedings of the 2009 International Joint Conference on Artificial IntelligenceCampus network’s Internet accessing traffic is complicated, non-linear and periodical. Our goal is to give out a engineering approach to prediction of network traffic based Time-series analysis model(EPTS) for campus exit-link. In our EPTS with rate-...
Attention-based spatial-temporal graph transformer for traffic flow forecasting
AbstractTraffic forecasting is significant for establishing intelligent traffic systems. However, the complex spatial-temporal relationships of traffic flow data make it challenging. Despite some studies investigating this problem, capturing complex ...
Comments
Information & Contributors
Information
Published In
March 2024
1246 pages
ISBN:9798400703713
DOI:10.1145/3616855
- General Chairs:
- Luz Angélica,
- Silvio Lattanzi,
- Andrés Muñoz Medina,
- Program Chairs:
- Leman Akoglu,
- Aristides Gionis,
- Sergei Vassilvitskii
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 March 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- Fundamental Research Funds for the Central Universities
- National Key R&D Program of China
- Beijing Natural Science Foundation
- S&T Program of Hebei
- CCF-DiDi GAIA Collaborative Research Funds for Young Scholars
Conference
WSDM '24: The 17th ACM International Conference on Web Search and Data Mining
March 4 - 8, 2024
Merida, Mexico
Acceptance Rates
Overall Acceptance Rate 498 of 2,863 submissions, 17%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 253Total Downloads
- Downloads (Last 12 months)253
- Downloads (Last 6 weeks)28
Reflects downloads up to 27 Jul 2024
Other Metrics
Citations
Cited By
View all- Peng HZhang JHuang XHao ZLi AYu ZYu P(undefined)Unsupervised Social Bot Detection via Structural Information TheoryACM Transactions on Information Systems10.1145/3660522
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in