research-article

NeuralReconciler for Hierarchical Time Series Forecasting

Author:

Shiyu WangAuthors Info & Claims

WSDM '24: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

Pages 731 - 739

https://doi.org/10.1145/3616855.3635806

Published: 04 March 2024 Publication History

Abstract

Time series forecasting has wide-ranging applications in business intelligence, including predicting logistics demand and estimating power consumption in a smart grid, which subsequently facilitates decision-making processes. In many real-world scenarios, such as department sales of multiple Walmart stores across different locations, time series data possess hierarchical structures with non-linear and non-Gaussian properties. Thus, the task of leveraging structural information among hierarchical time series while learning from non-linear correlations and non-Gaussian data distributions becomes crucial to enhance prediction accuracy. This paper proposes a novel approach named NeuralReconciler for Hierarchical Time Series (HTS) prediction through trainable attention-based reconciliation and Normalizing Flow (NF). The latter is used to approximate the complex (usually non-Gaussian) data distribution for multivariate time series forecasting. To reconcile the HTS data, a new flexible reconciliation strategy via the attention-based encoder-decoder neural network is proposed, which is distinct from current methods that rely on strong assumptions (e.g., all forecasts being unbiased estimates and the noise distribution being Gaussian). Furthermore, using the reparameterization trick, each independent component (i.e., forecasts via NF and attention-based reconciliation) is integrated into a trainable end-to-end model. Our proposed NeuralReconciler has been extensively experimented on real-world datasets and achieved consistent state-of-the-art performance compared to well-acknowledged and advanced baselines, with a 20% relative improvement on five benchmarks.

References

[1]

Matthias Anderer and Feng Li. 2021. Forecasting reconciliation with a top-down alignment of independent level forecasts. arXiv preprint arXiv:2103.08250 (2021).

[2]

Souhaib Ben Taieb and Bonsoo Koo. 2019. Regularized regression for hierarchical forecasting without unbiasedness conditions. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1337--1347.

Digital Library

[3]

Davide Burba and Trista Chen. 2021. A Trainable Reconciliation Method for Hierarchical Time-Series. arXiv preprint arXiv:2101.01329 (2021).

[4]

Robyn Bushell, Gary M Prosser, Herbert William Faulkner, and Jafar Jafari. 2001. Tourism research in Australia. Journal of travel research 39, 3 (2001), 323--326.

[5]

Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza, Max Mergenthaler-Canseco, and Artur Dubrawski. 2022. N-hits: Neural hierarchical interpolation for time series forecasting. arXiv preprint arXiv:2201.12886 (2022).

[6]

Giorgio Corani, Dario Azzimonti, Joao PSC Augusto, and Marco Zaffalon. 2020. Probabilistic Reconciliation of Hierarchical Forecast via Bayes' Rule. In ECML/PKDD (3). 211--226.

[7]

Marco Cuturi. 2011. Fast Global Alignment Kernels. In ICML. 929--936.

[8]

Abhimanyu Das, Weihao Kong, Biswajit Paria, and Rajat Sen. 2022. A Top-Down Approach to Hierarchically Coherent Probabilistic Forecasting. arXiv preprint arXiv:2204.10414 (2022).

[9]

Emmanuel de Bézenac, Syama Sundar Rangapuram, Konstantinos Benidis, Michael Bohlke-Schneider, Richard Kurle, Lorenzo Stella, Hilaf Hasson, Patrick Gallinari, and Tim Januschowski. 2020. Normalizing Kalman Filters for Multivariate Time Series Analysis. In NeurIPS.

[10]

Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016).

[11]

Xing Han, Sambarta Dasgupta, and Joydeep Ghosh. 2021a. Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Series. In International Conference on Artificial Intelligence and Statistics. PMLR, 190--198.

[12]

Xing Han, Sambarta Dasgupta, and Joydeep Ghosh. 2021b. Simultaneously reconciled quantile forecasting of hierarchically related time series. In International Conference on Artificial Intelligence and Statistics. PMLR, 190--198.

[13]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[14]

Ross Hollyman, Fotios Petropoulos, and Michael E Tipping. 2021. Understanding forecast reconciliation. European Journal of Operational Research 294, 1 (2021), 149--160.

[15]

Rob J Hyndman, Roman A Ahmed, George Athanasopoulos, and Han Lin Shang. 2011. Optimal combination forecasts for hierarchical time series. Computational statistics & data analysis 55, 9 (2011), 2579--2589.

[16]

Diederik P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039 (2018).

[17]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[18]

Ivan Kobyzev, Simon Prince, and Marcus Brubaker. 2020. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).

[19]

Australia Labour Force. 2021. Australian Bureau of Statistics. https://www.abs.gov.au/statistics/labour/employment-and-unemployment/labour-force-australia/latest-release

[20]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 95--104.

Digital Library

[21]

Mike Laszkiewicz, Johannes Lederer, and Asja Fischer. 2021. Copula-Based Normalizing Flows. arXiv preprint arXiv:2107.07352 (2021).

[22]

Zitao Liu, Yan Yan, and Milos Hauskrecht. 2018. A flexible forecasting framework for hierarchical time series with seasonal patterns: A case study of web traffic. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 889--892.

Digital Library

[23]

Duc-Hung Luong, Huu-Trung Thieu, Abdelkader Outtagarts, and Yacine Ghamri-Doudane. 2018. Predictive autoscaling orchestration for cloud-native telecom microservices. In 2018 IEEE 5G World Forum (5GWF). IEEE, 153--158.

[24]

James E Matheson and Robert L Winkler. 1976. Scoring rules for continuous probability distributions. Management science, Vol. 22, 10 (1976), 1087--1096.

[25]

Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR (2023).

[26]

Kin G Olivares, O Nganba Meetei, Ruijun Ma, Rohan Reddy, Mengfei Cao, and Lee Dicker. 2021. Probabilistic hierarchical forecasting with deep poisson mixtures. arXiv preprint arXiv:2110.13179 (2021).

[27]

Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2019. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437 (2019).

[28]

Anastasios Panagiotelis, Puwasala Gamakumara, George Athanasopoulos, Rob J Hyndman, et al. 2020. Probabilistic forecast reconciliation: Properties, evaluation and score optimisation. Monash econometrics and business statistics working paper series 26 (2020), 20.

[29]

George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. 2019. Normalizing flows for probabilistic modeling and inference. arXiv preprint arXiv:1912.02762 (2019).

[30]

George Papamakarios, Theo Pavlakou, and Iain Murray. 2017. Masked autoregressive flow for density estimation. arXiv preprint arXiv:1705.07057 (2017).

[31]

Biswajit Paria, Rajat Sen, Amr Ahmed, and Abhimanyu Das. 2021. Hierarchically regularized deep forecasting. arXiv preprint arXiv:2106.07630 (2021).

[32]

Mijung Park and Marcel Nassar. 2014. Variational Bayesian inference for forecasting hierarchical time series. In International conference on machine learning (ICML), workshop on divergence methods for probabilistic inference, Beijing. Citeseer.

[33]

Syama Sundar Rangapuram, Lucien D Werner, Konstantinos Benidis, Pedro Mercado, Jan Gasthaus, and Tim Januschowski. 2021. End-to-End Learning of Coherent Probabilistic Forecasts for Hierarchical Time Series. In International Conference on Machine Learning. PMLR, 8832--8843.

[34]

Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. 2021. Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting. arXiv preprint arXiv:2101.12072 (2021).

[35]

Kashif Rasul, Abdul-Saboor Sheikh, Ingmar Schuster, Urs Bergmann, and Roland Vollgraf. 2020. Multivariate probabilistic time series forecasting via conditioned normalizing flows. arXiv preprint arXiv:2002.06103 (2020).

[36]

David Salinas, Michael Bohlke-Schneider, Laurent Callot, Roberto Medico, and Jan Gasthaus. 2019. High-dimensional multivariate forecasting with low-rank gaussian copula processes. arXiv preprint arXiv:1910.03002 (2019).

[37]

David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181--1191.

[38]

Matthias Seeger, Syama Rangapuram, Yuyang Wang, David Salinas, Jan Gasthaus, Tim Januschowski, and Valentin Flunkert. 2017. Approximate Bayesian inference in linear state space models for intermittent demand forecasting at scale. arXiv preprint arXiv:1709.07638 (2017).

[39]

Esteban G Tabak and Cristina V Turner. 2013. A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics, Vol. 66, 2 (2013), 145--164.

[40]

Souhaib Ben Taieb, James W Taylor, and Rob J Hyndman. 2017. Coherent probabilistic forecasts for hierarchical time series. In International Conference on Machine Learning. PMLR, 3348--3357.

[41]

Souhaib Ben Taieb, James W Taylor, and Rob J Hyndman. 2021. Hierarchical probabilistic forecasting of electricity demand with smart meter data. J. Amer. Statist. Assoc. 116, 533 (2021), 27--43.

[42]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[43]

Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. 2023. MICN: Multi-scale Local and Global Context Modeling for Long-term Series Forecasting. ICLR (2023).

[44]

Shanika L Wickramasuriya, George Athanasopoulos, and Rob J Hyndman. 2019. Optimal forecast reconciliation for hierarchical and grouped time series through trace minimization. J. Amer. Statist. Assoc. 114, 526 (2019), 804--819.

[45]

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34 (2021), 22419--22430.

[46]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI.

[47]

Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. 2022a. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in Neural Information Processing Systems 35 (2022), 12677--12690.

[48]

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022b. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. arXiv preprint arXiv:2201.12740 (2022).

Cited By

Zou DLu WZhu ZLu XZhou JWang XLiu KWang KSun RWang H(2024)OptScaler: A Collaborative Framework for Robust Autoscaling in the CloudProceedings of the VLDB Endowment10.14778/3685800.368582917:12(4090-4103)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685829
Wang SChu ZSun YLiu YGuo YChen YJian HMa LLu XZhou JSerra ESpezzano F(2024)Multiscale Representation Enhanced Temporal Flow Fusion Model for Long-Term Workload ForecastingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680072(4948-4956)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3680072
Sun HTu WZhan JZhao W(2024)HierNBeats: Hierarchical Neural Basis Expansion Analysis for Hierarchical Time Series ForecastingArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72347-6_17(251-266)Online publication date: 17-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72347-6_17

Index Terms

NeuralReconciler for Hierarchical Time Series Forecasting

Recommendations

Flow-Based End-to-End Model for Hierarchical Time Series Forecasting via Trainable Attentive-Reconciliation
Database Systems for Advanced Applications
Abstract
Time Series (TS) is one of the most common data formats in modern world, which often takes hierarchical structures, and is normally complicated with non-Gaussian and non-linear properties. Many businesses rely on accurate TS forecasting, under ...
A machine learning approach for forecasting hierarchical time series
Highlights
- A novel strategy for forecasting hierarchical time series using is proposed.
- A ...
Abstract
In this paper, we propose a machine learning approach for forecasting hierarchical time series. When dealing with hierarchical time series, apart from generating accurate forecasts, one needs to select a suitable method for producing ...
Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Selecting the right set of hyperparameters is crucial in time series forecasting. The classical temporal cross-validation framework for hyperparameter optimization (HPO) often leads to poor test performance because of a possible mismatch between ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '24: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

March 2024

1246 pages

ISBN:9798400703713

DOI:10.1145/3616855

General Chairs:
Luz Angélica
Caudillo Mata (MDA Geointelligence)
,
Silvio Lattanzi
Google Research
,
Andrés Muñoz Medina
Google Research
,
Program Chairs:
Leman Akoglu
CMU
,
Aristides Gionis
KTH
,
Sergei Vassilvitskii
Google Research

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '24

Sponsor:

WSDM '24: The 17th ACM International Conference on Web Search and Data Mining

March 4 - 8, 2024

Merida, Mexico

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
235
Total Downloads

Downloads (Last 12 months)235
Downloads (Last 6 weeks)27

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zou DLu WZhu ZLu XZhou JWang XLiu KWang KSun RWang H(2024)OptScaler: A Collaborative Framework for Robust Autoscaling in the CloudProceedings of the VLDB Endowment10.14778/3685800.368582917:12(4090-4103)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.14778/3685800.3685829
Wang SChu ZSun YLiu YGuo YChen YJian HMa LLu XZhou JSerra ESpezzano F(2024)Multiscale Representation Enhanced Temporal Flow Fusion Model for Long-Term Workload ForecastingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680072(4948-4956)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3680072
Sun HTu WZhan JZhao W(2024)HierNBeats: Hierarchical Neural Basis Expansion Analysis for Hierarchical Time Series ForecastingArtificial Neural Networks and Machine Learning – ICANN 202410.1007/978-3-031-72347-6_17(251-266)Online publication date: 17-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72347-6_17

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents