MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attention
This work addresses incremental improvements in neural forecasting for demand prediction, targeting applications in supply chain and inventory management.
The paper tackles the problem of improving probabilistic demand forecasting accuracy by proposing novel Transformer-based enhancements, including context-aligned decoder-encoder attention, context-dependent positional encoding, and error-aware decoder-self attention, resulting in significant reductions in forecast variability.
Recent advances in neural forecasting have produced major improvements in accuracy for probabilistic demand prediction. In this work, we propose novel improvements to the current state of the art by incorporating changes inspired by recent advances in Transformer architectures for Natural Language Processing. We develop a novel decoder-encoder attention for context-alignment, improving forecasting accuracy by allowing the network to study its own history based on the context for which it is producing a forecast. We also present a novel positional encoding that allows the neural network to learn context-dependent seasonality functions as well as arbitrary holiday distances. Finally we show that the current state of the art MQ-Forecaster (Wen et al., 2017) models display excess variability by failing to leverage previous errors in the forecast to improve accuracy. We propose a novel decoder-self attention scheme for forecasting that produces significant improvements in the excess variation of the forecast.