TriGuard: Testing Model Safety with Attribution Entropy, Verification, and Drift
This addresses the challenge of model safety evaluation for researchers and practitioners in machine learning, though it appears incremental as it combines existing verification and attribution methods with new metrics.
The paper tackles the problem of ensuring neural network reliability under adversarial and distributional shifts by proposing TriGuard, a unified safety evaluation framework combining formal robustness verification, attribution entropy, and a novel Attribution Drift Score. Experiments across three datasets and five architectures show that TriGuard reveals mismatches between model accuracy and interpretability, and entropy-regularized training reduces explanation drift without performance loss.
Deep neural networks often achieve high accuracy, but ensuring their reliability under adversarial and distributional shifts remains a pressing challenge. We propose TriGuard, a unified safety evaluation framework that combines (1) formal robustness verification, (2) attribution entropy to quantify saliency concentration, and (3) a novel Attribution Drift Score measuring explanation stability. TriGuard reveals critical mismatches between model accuracy and interpretability: verified models can still exhibit unstable reasoning, and attribution-based signals provide complementary safety insights beyond adversarial accuracy. Extensive experiments across three datasets and five architectures show how TriGuard uncovers subtle fragilities in neural reasoning. We further demonstrate that entropy-regularized training reduces explanation drift without sacrificing performance. TriGuard advances the frontier in robust, interpretable model evaluation.