Using dynamic loss weighting to boost improvements in forecast stability
This work addresses forecast instability for time series forecasting practitioners, representing an incremental improvement over prior methods that used static weighting.
The paper tackles the problem of rolling origin forecast instability in time series forecasting by investigating dynamic loss weighting during training, showing that existing methods and a proposed extension (Task-Aware Random Weighting) can improve stability without compromising accuracy.
Rolling origin forecast instability refers to variability in forecasts for a specific period induced by updating the forecast when new data points become available. Recently, an extension to the N-BEATS model for univariate time series point forecasting was proposed to include forecast stability as an additional optimization objective, next to accuracy. It was shown that more stable forecasts can be obtained without harming accuracy by minimizing a composite loss function that contains both a forecast error and a forecast instability component, with a static hyperparameter to control the impact of stability. In this paper, we empirically investigate whether further improvements in stability can be obtained without compromising accuracy by applying dynamic loss weighting algorithms, which change the loss weights during training. We show that existing dynamic loss weighting methods can achieve this objective and provide insights into why this might be the case. Additionally, we propose an extension to the Random Weighting approach -- Task-Aware Random Weighting -- which also achieves this objective.