Aligned explanations in neural networks
This addresses the trustworthiness issue in neural network explanations for users in prediction tasks, though it is incremental as it builds on existing feature attribution paradigms.
The paper tackles the problem of feature attribution methods not being directly linked to model predictions, proposing PiNets as a pseudo-linear network framework to ensure explanatory alignment. It demonstrates PiNets on image classification and segmentation tasks, showing they produce faithful explanations across multiple criteria.
Feature attribution is the dominant paradigm for explaining deep neural networks. However, most existing methods only loosely reflect the model's prediction-making process, thereby merely white-painting the black box. We argue that explanatory alignment is a key aspect of trustworthiness in prediction tasks: explanations must be directly linked to predictions, rather than serving as post-hoc rationalizations. We present model readability as a design principle enabling alignment, and PiNets as a modeling framework to pursue it in a deep learning context. PiNets are pseudo-linear networks that produce instance-wise linear predictions in an arbitrary feature space, making them linearly readable. We illustrate their use on image classification and segmentation tasks, demonstrating how PiNets produce explanations that are faithful across multiple criteria in addition to alignment.