research-article

TTS-Norm: Forecasting Tensor Time Series via Multi-Way Normalization

Authors:

Xuan SongAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 18, Issue 1

Article No.: 3, Pages 1 - 25

https://doi.org/10.1145/3605894

Published: 10 August 2023 Publication History

Abstract

Tensor time series (TTS) data, a generalization of one-dimensional time series on a high-dimensional space, is ubiquitous in real-world applications. Compared to modeling time series or multivariate time series, which has received much attention and achieved tremendous progress in recent years, tensor time series has been paid less effort. However, properly coping with the TTS is a much more challenging task, due to its high-dimensional and complex inner structure. In this article, we start by revealing the structure of TTS data from afn statistical view of point. Then, in line with this analysis, we perform Tensor Time Series forecasting via a proposed Multi-way Normalization (TTS-Norm), which effectively disentangles multiple heterogeneous low-dimensional substructures from the original high-dimensional structure. Finally, we design a novel objective function for TTS forecasting, accounting for the numerical heterogeneity among different low-dimensional subspaces of TTS. Extensive experiments on two real-world datasets verify the superior performance of our proposed model.

References

[1]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450.

[2]

Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive graph convolutional recurrent network for traffic forecasting. In Proceedings of the 34th Conference on Neural Information Processing Systems.

[3]

Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271.

[4]

Christoph Bandt and Bernd Pompe. 2002. Permutation entropy: A natural complexity measure for time series. Physical Review Letters 88, 17 (2002), 174102.

[5]

Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-yan Liu, and Liwei Wang. 2021. Graphnorm: A principled approach to accelerating graph neural network training. In Proceedings of the International Conference on Machine Learning. PMLR, 1204–1215.

[6]

Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. 2020. Spectral temporal graph neural network for multivariate time-series forecasting. In Proceedings of the 34th Conference on Neural Information Processing Systems.

[7]

Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, and Liang Sun. 2022. Learning to rotate: Quaternion transformer for complicated periodical time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 146–156.

Digital Library

[8]

Lucas Deecke, Iain Murray, and Hakan Bilen. 2018. Mode normalization. arXiv:1810.05466.

[9]

Jinliang Deng, Xiusi Chen, Zipei Fan, Renhe Jiang, Xuan Song, and Ivor W Tsang. 2021. The pulse of urban transport: Exploring the co-evolving pattern for spatio-temporal forecasting. ACM Transactions on Knowledge Discovery from Data (TKDD) 15, 6 (2021), 1–25.

Digital Library

[10]

Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W. Tsang. 2021. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 269–278.

Digital Library

[11]

Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W. Tsang. 2022. A multi-view multi-task learning framework for multi-variate time series forecasting. arXiv:2109.01657.

[12]

Leyan Deng, Defu Lian, Zhenya Huang, and Enhong Chen. 2022. Graph convolutional adversarial networks for spatiotemporal anomaly detection. IEEE Transactions on Neural Networks and Learning Systems 33, 6 (2022), 2416–2428.

[13]

Shen Fang, Qi Zhang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2019. GSTNet: Global spatial-temporal network for traffic flow prediction. In IJCAI. 2286–2293.

[14]

Ziquan Fang, Lu Pan, Lu Chen, Yuntao Du, and Yunjun Gao. 2021. MDTP: A multi-source deep traffic prediction framework over spatio-temporal trajectory data. Proceedings of the VLDB Endowment 14, 8 (2021), 1289–1297.

Digital Library

[15]

Xinyu Gong, Wuyang Chen, Tianlong Chen, and Zhangyang Wang. 2022. Sandwich batch normalization: A drop-in replacement for feature distribution heterogeneity. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2494–2504.

[16]

Kan Guo, Yongli Hu, Yanfeng Sun, Sean Qian, Junbin Gao, and Baocai Yin. 2021. Hierarchical graph convolution network for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 151–159.

[17]

Shengnan Guo, Youfang Lin, Shijie Li, Zhaoming Chen, and Huaiyu Wan. 2019. Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems 20, 10 (2019), 3913–3926.

[18]

Liangzhe Han, Bowen Du, Leilei Sun, Yanjie Fu, Yisheng Lv, and Hui Xiong. 2021. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 547–555.

Digital Library

[19]

Chao Huang, Chuxu Zhang, Peng Dai, and Liefeng Bo. 2021. Cross-interaction hierarchical attention networks for urban anomaly prediction. In Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence. 4359–4365.

[20]

Chao Huang, Chuxu Zhang, Jiashu Zhao, Xian Wu, Dawei Yin, and Nitesh Chawla. 2019. Mist: A multiview and multimodal spatial-temporal learning framework for citywide abnormal event forecasting. In Proceedings of the World Wide Web Conference. 717–728.

Digital Library

[21]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning. PMLR, 448–456.

Digital Library

[22]

Renhe Jiang, Zekun Cai, Zhaonan Wang, Chuang Yang, Zipei Fan, Quanjun Chen, Kota Tsubouchi, Xuan Song, and Ryosuke Shibasaki. 2021. Deepcrowd: A deep model for large-scale citywide crowd density and flow prediction. IEEE Transactions on Knowledge and Data Engineering 35, 1 (2021), 276–290.

[23]

Renhe Jiang, Xuan Song, Dou Huang, Xiaoya Song, Tianqi Xia, Zekun Cai, Zhaonan Wang, Kyoung-Sook Kim, and Ryosuke Shibasaki. 2019. Deepurbanevent: A system for predicting citywide crowd dynamics at big events. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2114–2122.

Digital Library

[24]

Renhe Jiang, Zhaonan Wang, Jiawei Yong, Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. 2022. Spatio-temporal meta-graph learning for traffic forecasting. Proceedings of the AAAI Conference on Artificial Intelligence 37, 7 (2022), 8078–8086.

[25]

Renhe Jiang, Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, and Ryosuke Shibasaki. 2021. Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 4515–4525.

Digital Library

[26]

Baoyu Jing, Hanghang Tong, and Yada Zhu. 2021. Network of tensor time series. In Proceedings of the Web Conference 2021. 2425–2437.

Digital Library

[27]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 95–104.

Digital Library

[28]

He Li, Xuejiao Li, Liangcai Su, Duo Jin, Jianbin Huang, and Deshuang Huang. 2022. Deep spatio-temporal adaptive 3d convolutional neural networks for traffic flow prediction. ACM Transactions on Intelligent Systems and Technology (TIST) 13, 2 (2022), 1–21.

Digital Library

[29]

Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Proceedings of the 33rd Conference on Neural Information Processing Systems.

[30]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proceedings of the International Conference on Learning Representations.

[31]

Defu Lian, Yongji Wu, Yong Ge, Xing Xie, and Enhong Chen. 2020. Geography-aware sequential location recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2009–2019.

Digital Library

[32]

Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. 2019. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1720–1730.

Digital Library

[33]

Mark Rogers, Lei Li, and Stuart J. Russell. 2013. Multilinear dynamical systems for tensor time series. In Proceedings of the 27th Conference on Neural Information Processing Systems.

[34]

Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu, Xin Cao, and Christian S. Jensen. 2022. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. Proc. VLDB Endow. 15, 11 (July 2022), 2733–2746.

Digital Library

[35]

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv:1607.08022.

[36]

Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew W. Senior, and Koray Kavukcuoglu. 2016. WaveNet: A generative model for raw audio. In SSW.

[37]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems.

[38]

Zhaonan Wang, Renhe Jiang, Zekun Cai, Zipei Fan, Xin Liu, Kyoung-Sook Kim, Xuan Song, and Ryosuke Shibasaki. 2021. Spatio-temporal-categorical graph neural networks for fine-grained multi-incident co-prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2060–2069.

Digital Library

[39]

Zhaonan Wang, Renhe Jiang, Hao Xue, Flora D Salim, Xuan Song, and Ryosuke Shibasaki. 2022. Event-aware multimodal mobility nowcasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 4228–4236.

[40]

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Proceedings of the 35th Conference on Neural Information Processing Systems.

[41]

Xian Wu, Chao Huang, Chuxu Zhang, and Nitesh V Chawla. 2020. Hierarchically structured transformer networks for fine-grained spatial event forecasting. In Proceedings of the Web Conference 2020. 2320–2330.

Digital Library

[42]

Yuxin Wu and Kaiming He. 2018. Group normalization. In Proceedings of the European Conference on Computer Vision (ECCV). 3–19.

Digital Library

[43]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 753–763.

Digital Library

[44]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence.

[45]

Song Yang, Jiamou Liu, and Kaiqi Zhao. 2021. Space meets time: Local spacetime neural network for traffic flow forecasting. In Proceedings of the 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 817–826.

[46]

Junchen Ye, Leilei Sun, Bowen Du, Yanjie Fu, Xinran Tong, and Hui Xiong. 2019. Co-prediction of multiple transportation demands based on deep spatio-temporal neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 305–313.

Digital Library

[47]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence.

[48]

Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. 2022. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8980–8987.

[49]

Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.

[50]

Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 1234–1241.

[51]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11106–11115.

[52]

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of the International Conference on Machine Learning. PMLR, 27268–27286.

[53]

Xian Zhou, Yanyan Shen, Yanmin Zhu, and Linpeng Huang. 2018. Predicting multi-step citywide passenger demands using attention-based neural networks. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 736–744.

Digital Library

Cited By

Xi PCheng DLu GDeng ZZhang GZhang S(2025)Identifying local useful information for attribute graph anomaly detectionNeurocomputing10.1016/j.neucom.2024.128900617(128900)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.128900
Wang ZLin YLi SHan Y(2024)TFE-DATT: A Multi-dimensional Feature Learning Network for Spectrum Tensor Forecasting2024 IEEE/CIC International Conference on Communications in China (ICCC)10.1109/ICCC62479.2024.10681836(1982-1987)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICCC62479.2024.10681836
Li WZhu Y(2024)Forecasting Urban Agglomeration Air Quality: A Data-Driven Model With the Gaussian Decoupled Representation ExtractorIEEE Access10.1109/ACCESS.2024.351219912(183103-183116)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3512199
Show More Cited By

Index Terms

TTS-Norm: Forecasting Tensor Time Series via Multi-Way Normalization
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications
    1. Data mining
    2. Spatial-temporal systems

Recommendations

Nonlocal image denoising via adaptive tensor nuclear norm minimization

Nonlocal self-similarity shows great potential in image denoising. Therefore, the denoising performance can be attained by accurately exploiting the nonlocal prior. In this paper, we model nonlocal similar patches through the multi-linear approach and ...
Robust low tubal rank tensor completion via factor tensor norm minimization
Highlights
- We give the definitions of tensor double norm and tensor Frobenius/nuclear hybrid norm, and regard them as low-rank regularization penalty of tensor ...
Abstract
Recent research has demonstrated that low tubal rank recovery based on tensor has received extensive attention. In this correspondence, we define tensor double nuclear norm and tensor Frobenius/nuclear hybrid norm to induce a surrogate ...
Tensor completion via multi-directional partial tensor nuclear norm with total variation regularization
Abstract
This paper addresses the tensor completion problem, whose task is to estimate missing values with limited information. However, the crux of this problem is how to reasonably represent the low-rank structure embedded in the underlying data. In this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 18, Issue 1

January 2024

854 pages

EISSN:1556-472X

DOI:10.1145/3613504

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 August 2023

Online AM: 26 June 2023

Accepted: 19 June 2023

Revised: 15 March 2023

Received: 01 November 2022

Published in TKDD Volume 18, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Project
Guangdong Provincial Key Laboratory

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
849
Total Downloads

Downloads (Last 12 months)417
Downloads (Last 6 weeks)32

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xi PCheng DLu GDeng ZZhang GZhang S(2025)Identifying local useful information for attribute graph anomaly detectionNeurocomputing10.1016/j.neucom.2024.128900617(128900)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.128900
Wang ZLin YLi SHan Y(2024)TFE-DATT: A Multi-dimensional Feature Learning Network for Spectrum Tensor Forecasting2024 IEEE/CIC International Conference on Communications in China (ICCC)10.1109/ICCC62479.2024.10681836(1982-1987)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICCC62479.2024.10681836
Li WZhu Y(2024)Forecasting Urban Agglomeration Air Quality: A Data-Driven Model With the Gaussian Decoupled Representation ExtractorIEEE Access10.1109/ACCESS.2024.351219912(183103-183116)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3512199
Schosser J(2024)Beyond point forecasts: Uncertainty quantification in tensor extrapolation for relational time series dataCommunications in Statistics: Case Studies, Data Analysis and Applications10.1080/23737484.2024.232917110:2(107-124)Online publication date: 19-Mar-2024
https://doi.org/10.1080/23737484.2024.2329171

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents