Are Data-driven Explanations Robust against Out-of-distribution Data?
This addresses a critical issue for users of high-stakes black-box models by enhancing explanation reliability under distribution shifts, though it is incremental as it builds on existing explanation methods.
The paper tackles the problem of data-driven explanation methods being unreliable under out-of-distribution data, showing that models can predict correctly but yield poor explanations, and proposes a model-agnostic framework called Distributionally Robust Explanations (DRE) that significantly improves both explanation and prediction robustness against distributional shifts.
As black-box models increasingly power high-stakes applications, a variety of data-driven explanation methods have been introduced. Meanwhile, machine learning models are constantly challenged by distributional shifts. A question naturally arises: Are data-driven explanations robust against out-of-distribution data? Our empirical results show that even though predict correctly, the model might still yield unreliable explanations under distributional shifts. How to develop robust explanations against out-of-distribution data? To address this problem, we propose an end-to-end model-agnostic learning framework Distributionally Robust Explanations (DRE). The key idea is, inspired by self-supervised learning, to fully utilizes the inter-distribution information to provide supervisory signals for the learning of explanations without human annotation. Can robust explanations benefit the model's generalization capability? We conduct extensive experiments on a wide range of tasks and data types, including classification and regression on image and scientific tabular data. Our results demonstrate that the proposed method significantly improves the model's performance in terms of explanation and prediction robustness against distributional shifts.