I-trustworthy Models. A framework for trustworthiness evaluation of probabilistic classifiers
This work addresses the need for better trustworthiness assessment in probabilistic models used in society and science, though it is incremental as it builds on existing calibration concepts.
The authors tackled the problem of evaluating trustworthiness in probabilistic classifiers by proposing the I-trustworthy framework, which links local calibration to trustworthiness and uses a kernel-based test statistic (KLCE) with theoretical guarantees, demonstrating its effectiveness on simulated and real-world datasets while showing that existing recalibration methods are insufficient.
As probabilistic models continue to permeate various facets of our society and contribute to scientific advancements, it becomes a necessity to go beyond traditional metrics such as predictive accuracy and error rates and assess their trustworthiness. Grounded in the competence-based theory of trust, this work formalizes I-trustworthy framework -- a novel framework for assessing the trustworthiness of probabilistic classifiers for inference tasks by linking local calibration to trustworthiness. To assess I-trustworthiness, we use the local calibration error (LCE) and develop a method of hypothesis-testing. This method utilizes a kernel-based test statistic, Kernel Local Calibration Error (KLCE), to test local calibration of a probabilistic classifier. This study provides theoretical guarantees by offering convergence bounds for an unbiased estimator of KLCE. Additionally, we present a diagnostic tool designed to identify and measure biases in cases of miscalibration. The effectiveness of the proposed test statistic is demonstrated through its application to both simulated and real-world datasets. Finally, LCE of related recalibration methods is studied, and we provide evidence of insufficiency of existing methods to achieve I-trustworthiness.