Resilient neural forecasting systems

M Bohlke-Schneider, S Kapoor… - Proceedings of the Fourth …, 2020 - dl.acm.org
Proceedings of the Fourth International Workshop on Data Management for End …, 2020dl.acm.org
Industrial machine learning systems face data challenges that are often under-explored in
the academic literature. Common data challenges are data distribution shifts, missing values
and anomalies. In this paper, we discuss data challenges and solutions in the context of a
Neural Forecasting application on labor planning. We discuss how to make this forecasting
system resilient to these data challenges. We address changes in data distribution with a
periodic retraining scheme and discuss the critical importance of model stability in this …
Industrial machine learning systems face data challenges that are often under-explored in the academic literature. Common data challenges are data distribution shifts, missing values and anomalies. In this paper, we discuss data challenges and solutions in the context of a Neural Forecasting application on labor planning. We discuss how to make this forecasting system resilient to these data challenges. We address changes in data distribution with a periodic retraining scheme and discuss the critical importance of model stability in this setting. Furthermore, we show how our deep learning model deals with missing values natively without requiring imputation. Finally, we describe how we detect anomalies in the input data and mitigate their effect before they impact the forecasts. This results in a fully autonomous forecasting system that compares favourably to a hybrid system consisting of the algorithm and human overrides.
ACM Digital Library