Resilient Neural Forecasting Systems
This addresses practical data reliability issues in industrial forecasting applications, though it appears incremental in applying known solutions to a specific domain.
The paper tackles data challenges in industrial neural forecasting systems by implementing a resilient approach that handles distribution shifts, missing values, and anomalies, resulting in a fully autonomous system that outperforms a hybrid system with human overrides.
Industrial machine learning systems face data challenges that are often under-explored in the academic literature. Common data challenges are data distribution shifts, missing values and anomalies. In this paper, we discuss data challenges and solutions in the context of a Neural Forecasting application on labor planning.We discuss how to make this forecasting system resilient to these data challenges. We address changes in data distribution with a periodic retraining scheme and discuss the critical importance of model stability in this setting. Furthermore, we show how our deep learning model deals with missing values natively without requiring imputation. Finally, we describe how we detect anomalies in the input data and mitigate their effect before they impact the forecasts. This results in a fully autonomous forecasting system that compares favorably to a hybrid system consisting of the algorithm and human overrides.