Calibration Meets Explanation: A Simple and Effective Approach for Model Confidence Estimates
This work addresses the need for more trustworthy black-box models in machine learning by enhancing confidence estimates, though it appears incremental as it builds on existing calibration techniques.
The paper tackles the problem of improving model confidence calibration by leveraging model explanations, showing that their method CME reduces expected calibration errors across six datasets and two pre-trained language models in both in-domain and out-of-domain settings.
Calibration strengthens the trustworthiness of black-box models by producing better accurate confidence estimates on given examples. However, little is known about if model explanations can help confidence calibration. Intuitively, humans look at important features attributions and decide whether the model is trustworthy. Similarly, the explanations can tell us when the model may or may not know. Inspired by this, we propose a method named CME that leverages model explanations to make the model less confident with non-inductive attributions. The idea is that when the model is not highly confident, it is difficult to identify strong indications of any class, and the tokens accordingly do not have high attribution scores for any class and vice versa. We conduct extensive experiments on six datasets with two popular pre-trained language models in the in-domain and out-of-domain settings. The results show that CME improves calibration performance in all settings. The expected calibration errors are further reduced when combined with temperature scaling. Our findings highlight that model explanations can help calibrate posterior estimates.