research-article

Multi-Granularity Residual Learning with Confidence Estimation for Time Series Prediction

Authors:

Enhong Chen, and

Jiang BianAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

Pages 112 - 121

https://doi.org/10.1145/3485447.3512056

Published: 25 April 2022 Publication History

Abstract

Time-series prediction is of high practical value in a wide range of applications such as econometrics and meteorology, where the data are commonly formed by temporal patterns. Most prior works ignore the diversity of dynamic pattern frequency, i.e., different granularities, suffering from insufficient information exploitation. Thus, multi-granularity learning is still under-explored for time-series prediction. In this paper, we propose a Multi-granularity Residual Learning Framework (MRLF) for more effective time series prediction. For a given time series, intuitively, there are more or less semantic overlaps and validity differences among its representations of different granularities. Due to the information redundancy, straightforward methods that leverage multi-granularity data, such as concatenation or ensemble, can easily lead to the model being dominated by the redundant coarse-grained trend information. Therefore, we design a novel residual learning net to model the prior knowledge of the fine-grained data’s distribution through the coarse-grained one. Then, by calculating the residual between multi-granularity data, the redundant information be removed. Furthermore, to alleviate the side effect of validity differences, we introduce a self-supervised objective for confidence estimation, which delivers more effective optimization without the requirement of additional annotation efforts. Extensive experiments on the real-world datasets indicate that multi-granular information significantly improves the time series prediction performance, and our model is superior in capturing such information.

References

[1]

2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181–1191.

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (2015).

[3]

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In International Conference on Machine Learning. 531–540.

[4]

Mikolaj Binkowski, Gautier Marti, and Philippe Donnat. 2018. Autoregressive convolutional neural networks for asynchronous time series. In International Conference on Machine Learning. PMLR, 580–589.

[5]

George Box. 2013. Box and Jenkins: time series analysis, forecasting and control. In A Very British Affair. Springer, 161–215.

[6]

Atul J Butte and Isaac S Kohane. 1999. Mutual information relevance networks: functional genomic clustering using pairwise entropy measurements. In Biocomputing 2000. World Scientific, 418–429.

[7]

Chi Chen, Li Zhao, Jiang Bian, Chunxiao Xing, and Tie-Yan Liu. 2019. Investment behaviors can tell what inside: Exploring stock intrinsic properties for stock trend prediction. In Proceedings of the 25th ACM SIGKDD. 2376–2384.

Digital Library

[8]

Mingyue Cheng, Fajie Yuan, Qi Liu, Xin Xin, and Enhong Chen. 2021. Learning Transferable User Representations with Sequential Behaviors via Contrastive Pre-training. In 2021 IEEE International Conference on Data Mining (ICDM). 51–60.

[9]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078(2014).

[10]

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555(2014).

[11]

Zhicheng Cui, Wenlin Chen, and Yixin Chen. 2016. Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995(2016).

[12]

Shumin Deng, Ningyu Zhang, Wen Zhang, Jiaoyan Chen, Jeff Z Pan, and Huajun Chen. 2019. Knowledge-driven stock trend prediction and explanation via temporal convolutional network. In Companion Proceedings of The 2019 World Wide Web Conference. 678–685.

Digital Library

[13]

Yi Ding, Weiqing Liu, Jiang Bian, Daoqiang Zhang, and Tie-Yan Liu. 2018. Investor-imitator: A framework for trading knowledge extraction. In Proceedings of the 24th ACM SIGKDD. 1310–1319.

Digital Library

[14]

Fuli Feng, Huimin Chen, Xiangnan He, Ji Ding, Maosong Sun, and Tat-Seng Chua. 2019. Enhancing Stock Movement Prediction with Adversarial Training. IJCAI (2019).

[15]

Pierre Geurts 2018. dynGENIE3: dynamical GENIE3 for the inference of gene networks from time series expression data. Scientific reports 8, 1 (2018), 1–12.

[16]

Andrew C Harvey. 1990. Forecasting, structural time series models and the Kalman filter. (1990).

[17]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[18]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In European conference on computer vision. Springer, 630–645.

[19]

Steven Craig Hillmer and George C Tiao. 1982. An ARIMA-model-based approach to seasonal adjustment. J. Amer. Statist. Assoc. 77, 377 (1982), 63–70.

[20]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.

Digital Library

[21]

Arthur E Hoerl and Robert W Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12, 1 (1970), 55–67.

[22]

Min Hou, Chang Xu, Yang Liu, Weiqing Liu, Jiang Bian, Le Wu, Zhi Li, Enhong Chen, and Tie-Yan Liu. 2021. Stock Trend Prediction with Multi-Granularity Data: A Contrastive Learning Approach with Adaptive Fusion. Association for Computing Machinery, New York, NY, USA, 700–709.

[23]

Siteng Huang, Donglin Wang, Xuehan Wu, and Ao Tang. 2019. Dsanet: Dual self-attention network for multivariate time series forecasting. In Proceedings of the 28th ACM international conference on information and knowledge management. 2129–2132.

Digital Library

[24]

J Stuart Hunter. 1986. The exponentially weighted moving average. Journal of quality technology 18, 4 (1986), 203–210.

[25]

Diederick P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR).

[26]

Justin B Kinney and Gurinder S Atwal. 2014. Equitability, mutual information, and the maximal information coefficient. Proceedings of the National Academy of Sciences 111, 9 (2014), 3354–3359.

[27]

Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The Efficient Transformer. In International Conference on Learning Representations.

[28]

Alireza Koochali, Peter Schichtel, Andreas Dengel, and Sheraz Ahmed. 2019. Probabilistic forecasting of sensory data with generative adversarial networks–forgan. IEEE Access 7(2019), 63868–63880.

[29]

Nojun Kwak and Chong-Ho Choi. 2002. Input feature selection by mutual information based on Parzen window. IEEE transactions on pattern analysis and machine intelligence 24, 12(2002), 1667–1671.

Digital Library

[30]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 95–104.

Digital Library

[31]

Zhi Li, Hongke Zhao, Qi Liu, Zhenya Huang, Tao Mei, and Enhong Chen. 2018. Learning from History and Present: Next-Item Recommendation via Discriminatively Exploiting User Behaviors. In Proceedings of the 24th ACM SIGKDD(KDD ’18). New York, NY, USA, 1734–1743.

Digital Library

[32]

Guang Liu, Yuzhao Mao, Qi Sun, Hailong Huang, Weiguo Gao, Xuan Li, Jianping Shen, Ruifan Li, and Xiaojie Wang. 2020. Multi-scale Two-way Deep Neural Network for Stock Trend Prediction. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence,IJCAI-20. 4555–4561.

[33]

Frederik Maes, Andre Collignon, Dirk Vandermeulen, Guy Marchal, and Paul Suetens. 1997. Multimodality image registration by maximization of mutual information. IEEE transactions on Medical Imaging 16, 2 (1997), 187–198.

[34]

Manfred Mudelsee. 2013. Climate time series analysis. Vol. 30. Springer.

[35]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748(2018).

[36]

Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2019. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437(2019).

[37]

Clare Ostle, Richard C Thompson, Derek Broughton, Lance Gregory, Marianne Wootton, and David G Johns. 2019. The rise in ocean plastics evidenced from a 60-year time series. Nature communications 10, 1 (2019), 1–6.

[38]

Hanchuan Peng, Fuhui Long, and Chris Ding. 2005. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on pattern analysis and machine intelligence 27, 8(2005), 1226–1238.

Digital Library

[39]

Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison Cottrell. 2017. A dual-stage attention-based recurrent neural network for time series prediction. IJCAI (2017), 2627–2633.

Digital Library

[40]

Marco S. Reis. 2019. Multiscale and Multi-Granularity Process Analytics: A Review. Processes 7, 2 (2019). https://doi.org/10.3390/pr7020061

[41]

Stephen Roberts, Michael Osborne, Mark Ebden, Steven Reece, Neale Gibson, and Suzanne Aigrain. 2013. Gaussian processes for time-series modelling. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 371, 1984 (2013), 20110550.

[42]

Shun-Yao Shih, Fan-Keng Sun, and Hung-yi Lee. 2019. Temporal pattern attention for multivariate time series forecasting. Machine Learning 108, 8 (2019), 1421–1441.

Digital Library

[43]

Arunesh Kumar Singh, S Khatoon Ibraheem, Md Muazzam, and DK Chaturvedi. 2013. An overview of electricity demand forecasting techniques. Network and Complex Systems 3, 3 (2013), 38–48.

[44]

Huan Song, Deepta Rajan, Jayaraman J Thiagarajan, and Andreas Spanias. 2018. Attend and diagnose: Clinical time series analysis using attention models. In Thirty-second AAAI conference on artificial intelligence.

[45]

Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. 2020. Joint modeling of local and global temporal dynamics for multivariate time series forecasting with missing values. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5956–5963.

[46]

Naftali Tishby, Fernando C Pereira, and William Bialek. 2000. The information bottleneck method. arXiv preprint physics/0004057(2000).

[47]

Vladimir Vapnik, Steven E Golowich, Alex Smola, 1997. Support vector method for function approximation, regression estimation, and signal processing. Advances in neural information processing systems (1997), 281–287.

[48]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.

[49]

Likang Wu, Zhi Li, Hongke Zhao, Qi Liu, and Enhong Chen. 2022. Estimating Fund-Raising Performance for Start-up Projects from a Market Graph Perspective. Pattern Recogn. 121, C (jan 2022), 13 pages.

[50]

Likang Wu, Zhi Li, Hongke Zhao, Zhen Pan, Qi Liu, and Enhong Chen. 2020. Estimating early fundraising performance of innovations via graph-based market environment model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6396–6403.

[51]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD. 753–763.

Digital Library

[52]

Wentao Xu, Weiqing Liu, Chang Xu, Jiang Bian, Jian Yin, and Tie-Yan Liu. 2021. REST: Relational Event-driven Stock Trend Forecasting. In WWW ’21: The Web Conference 2021. ACM / IW3C2, 1–10. https://doi.org/10.1145/3442381.3450032

Digital Library

[53]

Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. 2017. Stock price prediction via discovering multi-frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD. 2141–2149.

Digital Library

[54]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI.

Cited By

Chen HEldardiry H(2024)Graph Time-series Modeling in Deep Learning: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/363853418:5(1-35)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3638534
Liu XHu JLi YDiao SLiang YHooi BZimmermann RChua TNgo CKa-Wei Lee RKumar RLauw H(2024)UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series ForecastingProceedings of the ACM on Web Conference 202410.1145/3589334.3645434(4095-4106)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645434
Zhou CChe CWang PZhang Q(2024)DiformerKnowledge-Based Systems10.1016/j.knosys.2023.111061281:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111061
Show More Cited By

Index Terms

Multi-Granularity Residual Learning with Confidence Estimation for Time Series Prediction

Index terms have been assigned to the content through auto-classification.

Recommendations

PLAE: Time-Series Prediction Improvement by Adaptive Decomposition
PRICAI 2022: Trends in Artificial Intelligence
Abstract
Univariate time-series forecasting is a kind of commonly encountered yet tough problem. Most of the forecast algorithms’ performance is constrained by the limited information due to the single input dimension. No matter how capable a forecast ...
Read More
Comparison of Predictive Models for Forecasting Time-series Data
ICBDR '19: Proceedings of the 3rd International Conference on Big Data Research

Dramatic increase in data size enabled researchers to study analysis and prediction of big data. Big data can be formed in many ways and one alternative is through the use of sensors. An important aspect of data coming from sensors is that they are time-...
Read More
DPHM-Net:de-redundant multi-period hybrid modeling network for long-term series forecasting
Abstract
Deep learning models have been widely applied in the field of long-term forecasting has achieved significant success, with the incorporation of inductive bias such as periodicity to model multi-granularity representations of time series being a ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,074
Total Downloads

Downloads (Last 12 months)273
Downloads (Last 6 weeks)13

Other Metrics

View Author Metrics

Citations

Cited By

Chen HEldardiry H(2024)Graph Time-series Modeling in Deep Learning: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/363853418:5(1-35)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3638534
Liu XHu JLi YDiao SLiang YHooi BZimmermann RChua TNgo CKa-Wei Lee RKumar RLauw H(2024)UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series ForecastingProceedings of the ACM on Web Conference 202410.1145/3589334.3645434(4095-4106)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645434
Zhou CChe CWang PZhang Q(2024)DiformerKnowledge-Based Systems10.1016/j.knosys.2023.111061281:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111061
Liu ZCheng MLi ZHuang ZLiu QXie YChen EOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Adaptive normalization for non-stationary time series forecastingProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666750(14273-14292)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666750
Cheng MLiu QLiu ZLi ZLuo YChen E(2023)FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series ClassificationProceedings of the ACM Web Conference 202310.1145/3543507.3583205(1437-1445)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583205
Wang MChen FGuo JJia W(2023)Improving Stock Trend Prediction with Multi-granularity Denoising Contrastive Learning2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191523(1-10)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191523
Gao YHuang XZhou XGao XLi GChen G(2023)DBAugur: An Adversarial-based Trend Forecasting System for Diversified Workloads2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00385(27-39)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00385
Wang MWang SGuo JJia W(2023)Improving stock trend prediction with pretrain multi-granularity denoising contrastive learningKnowledge and Information Systems10.1007/s10115-023-02006-1Online publication date: 28-Dec-2023
https://doi.org/10.1007/s10115-023-02006-1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents