What went wrong and when? Instance-wise Feature Importance for Time-series Models
This addresses the need for interpretability in time-series models, particularly in healthcare, though it is incremental as it builds on existing explanation methods.
The paper tackles the problem of explaining time-series models for high-stakes applications like healthcare by proposing FIT, a framework that evaluates observation importance by quantifying shifts in predictive distributions, and demonstrates superiority over baselines on clinical data.
Explanations of time series models are useful for high stakes applications like healthcare but have received little attention in machine learning literature. We propose FIT, a framework that evaluates the importance of observations for a multivariate time-series black-box model by quantifying the shift in the predictive distribution over time. FIT defines the importance of an observation based on its contribution to the distributional shift under a KL-divergence that contrasts the predictive distribution against a counterfactual where the rest of the features are unobserved. We also demonstrate the need to control for time-dependent distribution shifts. We compare with state-of-the-art baselines on simulated and real-world clinical data and demonstrate that our approach is superior in identifying important time points and observations throughout the time series.