Model Explanations under Calibration
This addresses the problem of unreliable explanations in recommender systems for users and developers, but it is incremental as it builds on existing calibration and interpretability research.
The paper studied how model calibration affects the stability of attention-based explanations in recommender systems, finding that attention distributions are highly unstable for un-calibrated models, raising doubts about their utility for explainability.
Explaining and interpreting the decisions of recommender systems are becoming extremely relevant both, for improving predictive performance, and providing valid explanations to users. While most of the recent interest has focused on providing local explanations, there has been a much lower emphasis on studying the effects of model dynamics and its impact on explanation. In this paper, we perform a focused study on the impact of model interpretability in the context of calibration. Specifically, we address the challenges of both over-confident and under-confident predictions with interpretability using attention distribution. Our results indicate that the means of using attention distributions for interpretability are highly unstable for un-calibrated models. Our empirical analysis on the stability of attention distribution raises questions on the utility of attention for explainability.