Challenges of Explainability in Continual Learning for Time Series Forecasting
For researchers in continual learning and time series forecasting, this work provides insights into using explainability to analyze model behavior during adaptation, but the results are qualitative and lack concrete performance numbers.
The paper investigates using explainability methods (attention rollout, Grad-CAM) to understand continual learning with Experience Replay for time series forecasting on real-world piezometric data, revealing how attribution patterns evolve and can inform adaptation strategies.
Deep learning models have shown strong potential for time series forecasting, yet their deployment in real-world environmental monitoring remains challenging due to non-stationary dynamics and limited explainability. In this work, we investigate explainability as a central tool for understanding continual learning in adaptive time series forecasting, with Experience Replay strategies. We study neural forecasting architectures such as PatchMixer, PatchTST and DLinear, augmented with attention-based sampling mechanisms to support model adaptation over time. Explainability is leveraged through attention rollout and gradient-based attribution methods (Grad-CAM) to analyze both predictive behavior and sampling strategies within a continual learning framework. Experiments conducted on real-world piezometric time series exhibiting heterogeneous patterns and regime shifts show that analyzing model and sampling behaviors provides valuable insights into the dynamics of the continual learning framework. Beyond predictive performance, our results highlight the challenges and opportunities of using explainability to understand continual learning behaviors, revealing how attribution patterns evolve over time and how they can inform data selection and adaptation strategies in non-stationary forecasting scenarios.