research-article

Weakly Guided Adaptation for Robust Time Series Forecasting

Authors:

Christian S. JensenAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 17, Issue 4

Pages 766 - 779

https://doi.org/10.14778/3636218.3636231

Published: 01 December 2023 Publication History

Abstract

Robust multivariate time series forecasting is crucial in many cyberphysical and Internet of Things applications. Existing state-of-the-art robust forecasting models decompose time series into independent functions covering trends and periodicities. However, these independent functions fail to capture correlations among multiple time series, thereby reducing prediction accuracy. Moreover, existing robust forecasting models treat certain abrupt but normal changes, e.g., caused by holidays, as outliers because they occur infrequently and have data distributions that resemble those of outliers. This exacerbates model bias and reduces prediction accuracy. This paper aims to capture correlations across multiple time series and abrupt but normal changes, thereby improving prediction accuracy. We employ weak labels to partition the dataset into source and target domains. Then, we propose the Domain Adversarial Robust Forecaster (DARF). This forecasting model is based on adversarial domain adaptation and includes two novel modules: Correlated Robust Forecaster (CORF) and Domain Critic. Specifically, CORF constitutes an encoder-decoder framework proficient at robust multivariate time series forecasting, and Domain Critic works to reduce data bias. Extensive experiments and discussions show that DARF is capable of state-of-the-art forecasting accuracy.

References

[1]

Seungryul Baek, Kwang In Kim, and Tae-Kyun Kim. 2020. Weakly-Supervised Domain Adaptation via GAN and Mesh Model for Estimating 3D Hand Poses Interacting Objects. In CVPR. 6120--6130.

[2]

Ruichu Cai, Jiawei Chen, Zijian Li, Wei Chen, Keli Zhang, Junjian Ye, Zhuozhang Li, Xiaoyan Yang, and Zhenjie Zhang. 2021. Time Series Domain Adaptation via Sparse Associative Structure Alignment. In AAAI. 6859--6867. https://ojs.aaai.org/index.php/AAAI/article/view/16846

[3]

David Campos, Miao Zhang, Bin Yang, Tung Kieu, Chenjuan Guo, and Christian S. Jensen. 2023. LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation. Proc. ACM Manag. Data 1, 2 (2023), 171:1--171:27.

[4]

Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, and Liang Sun. 2022. Learning to Rotate: Quaternion Transformer for Complicated Periodical Time Series Forecasting. In KDD. 146--156.

Digital Library

[5]

Yuzhou Chen, Ignacio Segovia-Dominguez, and Yulia R. Gel. 2021. Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting. In ICML (Proceedings of Machine Learning Research), Vol. 139. 1684--1694. http://proceedings.mlr.press/v139/chen21o.html

[6]

Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. 2022. Towards Spatio-Temporal Aware Traffic Time Series Forecasting. In ICDE. 2900--2913.

[7]

Razvan-Gabriel Cirstea, Tung Kieu, Chenjuan Guo, Bin Yang, and Sinno Jialin Pan. 2021. EnhanceNet: Plugin Neural Networks for Enhancing Correlated Time Series Forecasting. In ICDE. 1739--1750.

[8]

Razvan-Gabriel Cirstea, Bin Yang, and Chenjuan Guo. 2019. Graph Attention Recurrent Neural Networks for Correlated Time Series Forecasting. In MileTS19@KDD.

[9]

Andrew A Cook, Göksel Mısırlı, and Zhong Fan. 2019. Anomaly detection for IoT time-series data: A survey. IEEE Internet of Things Journal 7, 7 (2019), 6481--6494.

[10]

Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017. Language Modeling with Gated Convolutional Networks. In ICML (Proceedings of Machine Learning Research), Vol. 70. 933--941. http://proceedings.mlr.press/v70/dauphin17a.html

[11]

Christos Faloutsos, Jan Gasthaus, Tim Januschowski, and Yuyang Wang. 2018. Forecasting big time series: old and new. Proceedings of the VLDB Endowment 11, 12 (2018), 2102--2105.

Digital Library

[12]

Jingkun Gao, Xiaomin Song, Qingsong Wen, Pichao Wang, Liang Sun, and Huan Xu. 2020. RobustTAD: Robust time series anomaly detection via decomposition and convolutional neural networks. KDD-MileTS (2020).

[13]

Xin Gao, Fang Deng, and Xianghu Yue. 2020. Data augmentation in fault diagnosis based on the Wasserstein generative adversarial network with gradient penalty. Neurocomputing 396 (2020), 487--494.

[14]

Hailin Hu, MingJian Tang, and Chengcheng Bai. 2020. DATSING: Data Augmented Time Series Forecasting with Adversarial Domain Adaptation. In CIKM. 2061--2064.

Digital Library

[15]

Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018. Cross-Domain Weakly-Supervised Object Detection Through Progressive Domain Adaptation. In CVPR. 5001--5009.

[16]

Xiaoyong Jin, Youngsuk Park, Danielle C. Maddix, Hao Wang, and Yuyang Wang. 2022. Domain Adaptation for Time Series Forecasting via Attention Sharing. In ICML, Vol. 162. 10280--10297. https://proceedings.mlr.press/v162/jin22d.html

[17]

Tung Kieu, Bin Yang, Chenjuan Guo, Christian S. Jensen, Yan Zhao, Feiteng Huang, and Kai Zheng. 2022. Robust and Explainable Autoencoders for Unsupervised Time Series Outlier Detection. In ICDE. 3038--3050.

[18]

Serkan Kiranyaz, Onur Avci, Osama Abdeljaber, Turker Ince, Moncef Gabbouj, and Daniel J. Inman. 2019. 1D Convolutional Neural Networks and Applications: A Survey. CoRR abs/1905.03554 (2019). arXiv:1905.03554 http://arxiv.org/abs/1905.03554

[19]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks. In SIGIR. 95--104.

Digital Library

[20]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In ICLR. https://openreview.net/forum?id=SJiHXGWAZ

[21]

Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019. Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding. CoRR abs/1904.09482 (2019). arXiv:1904.09482 http://arxiv.org/abs/1904.09482

[22]

Alexander Loginov, Malcolm I. Heywood, and Garnett Carl Wilson. 2016. Benchmarking a coevolutionary streaming classifier under the individual household electric power consumption dataset. In IJCNN. 2834--2841.

[23]

Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Zheng Kai, Feiteng Huang, Jiandong Xie, and Christian S. Jensen. 2024. A Unified Replay-based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data. ICDE (2024).

[24]

Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2020. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In ICLR. https://openreview.net/forum?id=r1ecqn4YwB

[25]

Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, and Christopher Mutschler. 2022. Domain Adaptation for Time-Series Classification to Mitigate Covariate Shift. In MM. 5934--5943.

Digital Library

[26]

Victor M Panaretos and Yoav Zemel. 2019. Statistical aspects of Wasserstein distances. Annual review of statistics and its application 6 (2019), 405--431.

[27]

Simon Aagaard Pedersen, Bin Yang, and Christian S. Jensen. 2020. Anytime Stochastic Routing with Hybrid Learning. Proc. VLDB Endow. 13, 9 (2020), 1555--1567.

Digital Library

[28]

Sanjay Purushotham, Wilka Carvalho, Tanachat Nilanon, and Yan Liu. 2017. Variational Recurrent Adversarial Deep Domain Adaptation. In ICLR. https://openreview.net/forum?id=rk9eAFcxg

[29]

Chao Shang, Jie Chen, and Jinbo Bi. 2021. Discrete Graph Structure Learning for Forecasting Multiple Time Series. In ICLR. https://openreview.net/forum?id=WEHSlH5mOk

[30]

Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. 2022. Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting. In KDD. 1567--1577.

Digital Library

[31]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579--2605.

[32]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Annual Conference on Neural Information Processing Systems. 5998--6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

[33]

Andreas Veit, Neil Alldrin, Gal Chechik, Ivan Krasin, Abhinav Gupta, and Serge J. Belongie. 2017. Learning from Noisy Large-Scale Datasets with Minimal Supervision. In CVPR. 6575--6583.

[34]

Qingsong Wen, Jingkun Gao, Xiaomin Song, Liang Sun, Huan Xu, and Shenghuo Zhu. 2019. RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series. In AAAI. 5409--5416.

[35]

Qingsong Wen, Kai He, Liang Sun, Yingying Zhang, Min Ke, and Huan Xu. 2021. RobustPeriod: Robust Time-Frequency Mining for Multiple Periodicity Detection. In SIGMOD. 2328--2337.

[36]

Qingsong Wen, Linxiao Yang, Tian Zhou, and Liang Sun. 2022. Robust time series analysis and applications: An industrial perspective. In KDD. 4836--4837.

[37]

Qingsong Wen, Zhe Zhang, Yan Li, and Liang Sun. 2020. Fast RobustSTL: Efficient and Robust Seasonal-Trend Decomposition for Time Series with Complex Patterns. In KDD. 2203--2213.

[38]

Garrett Wilson, Janardhan Rao Doppa, and Diane J. Cook. 2020. Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data. In KDD. 1768--1778.

Digital Library

[39]

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven C. H. Hoi. 2022. CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting. In ICLR. https://openreview.net/forum?id=PilZY3omXV2

[40]

Xinle Wu, Dalin Zhang, Chenjuan Guo, Chaoyang He, Bin Yang, and Christian S. Jensen. 2021. AutoCTS: Automated Correlated Time Series Forecasting. Proc. VLDB Endow. 15, 4 (2021), 971--983.

Digital Library

[41]

Xinle Wu, Dalin Zhang, Miao Zhang, Chenjuan Guo, Bin Yang, and Christian S. Jensen. 2023. AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting. Proc. ACM Manag. Data 1, 1 (2023), 97:1--97:26.

[42]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. In KDD. 753--763.

Digital Library

[43]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In IJCAI. 1907--1913.

[44]

Linxiao Yang, Qingsong Wen, Bo Yang, and Liang Sun. 2021. A Robust and Efficient Multi-Scale Seasonal-Trend Decomposition. In ICASSP. 5085--5089.

[45]

Chin-Chia Michael Yeh, Nickolas Kavantzas, and Eamonn J. Keogh. 2017. Matrix Profile IV: Using Weakly Labeled Time Series to Predict Outcomes. Proc. VLDB Endow. 10, 12 (2017), 1802--1812.

Digital Library

[46]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. In IJCAI. 3634--3640.

[47]

Hongyuan Yu, Ting Li, Weichen Yu, Jianguo Li, Yan Huang, Liang Wang, and Alex X. Liu. 2022. Regularized Graph Structure Learning with Semantic Knowledge for Multi-variates Time-Series Forecasting. In IJCAI. 2362--2368.

[48]

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2022. Are Transformers Effective for Time Series Forecasting? CoRR abs/2205.13504 (2022). arXiv:2205.13504

[49]

Hanbo Zhang, Yawen Wang, Peng Wang, and Wei Wang. 2017. Burst-Based Event Classification on Weakly Labeled Time Series Data of Sensors. In BigData Congress. 452--459.

[50]

Kai Zhao, Chenjuan Guo, Yunyao Cheng, Miao Zhang, Peng Han, and Bin Yang. 2024. Multiple Time Series Forecasting with Dynamic Graph Modeling. Proc. VLDB Endow. (2024).

[51]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In AAAI. 11106--11115. https://ojs.aaai.org/index.php/AAAI/article/view/17325

[52]

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. In ICML (Proceedings of Machine Learning Research), Vol. 162. 27268--27286. https://proceedings.mlr.press/v162/zhou22g.html

[53]

Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, and Jinsong Wang. 2018. Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training. In ECCV (Lecture Notes in Computer Science), Vol. 11207. 297--313.

Digital Library

Cited By

Campos DYang BKieu TZhang MGuo CJensen C(2024)QCore: Data-Efficient, On-Device Continual Calibration for Quantized ModelsProceedings of the VLDB Endowment10.14778/3681954.368195717:11(2708-2721)Online publication date: 30-Aug-2024
https://doi.org/10.14778/3681954.3681957
Qiu XHu JZhou LWu XDu JZhang BGuo CZhou AJensen CSheng ZYang B(2024)TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsProceedings of the VLDB Endowment10.14778/3665844.366586317:9(2363-2377)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.14778/3665844.3665863
Jensen CYang BGuo CHu JTorp K(2024)Routing with Massive Trajectory Data2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00442(5542-5547)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00442
Show More Cited By

Index Terms

Weakly Guided Adaptation for Robust Time Series Forecasting

Index terms have been assigned to the content through auto-classification.

Recommendations

Time series forecasting by a seasonal support vector regression model

The support vector regression (SVR) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the applications of SVR models in a seasonal time series forecasting has not been widely investigated. This ...
Combining seasonal ARIMA models with computational intelligence techniques for time series forecasting

Seasonal autoregressive integrated moving average (SARIMA) models form one of the most popular and widely used seasonal time series models over the past three decades. However, in several researches it has been argued that they have two basic ...
Forecasting Trends in Time Series

Most time series methods assume that any trend will continue unabated, regardless of the forecast lead time. But recent empirical findings suggest that forecast accuracy can be improved by either damping or ignoring altogether trends which have a low ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 17, Issue 4

December 2023

309 pages

ISSN:2150-8097

Editors:
Meihui Zhang
Beijing Institute of Technology
,
Cyrus Shahabi
University of Southern California

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 December 2023

Published in PVLDB Volume 17, Issue 4

Check for updates

Badges

Artifacts Available / v1.1

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
203
Total Downloads

Downloads (Last 12 months)203
Downloads (Last 6 weeks)36

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Campos DYang BKieu TZhang MGuo CJensen C(2024)QCore: Data-Efficient, On-Device Continual Calibration for Quantized ModelsProceedings of the VLDB Endowment10.14778/3681954.368195717:11(2708-2721)Online publication date: 30-Aug-2024
https://doi.org/10.14778/3681954.3681957
Qiu XHu JZhou LWu XDu JZhang BGuo CZhou AJensen CSheng ZYang B(2024)TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsProceedings of the VLDB Endowment10.14778/3665844.366586317:9(2363-2377)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.14778/3665844.3665863
Jensen CYang BGuo CHu JTorp K(2024)Routing with Massive Trajectory Data2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00442(5542-5547)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00442
Miao HZhao YGuo CYang BZheng KHuang FXie JJensen C(2024)A Unified Replay-Based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00085(1050-1062)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00085
Zhao SZhou XJin MHou ZYang CLi ZWen QWang YWen YYuan X(2024)Rethinking self-supervised learning for time series forecasting: A temporal perspectiveKnowledge-Based Systems10.1016/j.knosys.2024.112652305(112652)Online publication date: Dec-2024
https://doi.org/10.1016/j.knosys.2024.112652
Wu XWu XYang BZhou LGuo CQiu XHu JSheng ZJensen C(2024)AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecastingThe VLDB Journal10.1007/s00778-024-00872-x33:5(1743-1770)Online publication date: 30-Jul-2024
https://doi.org/10.1007/s00778-024-00872-x

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents