What made you do this? Understanding black-box decisions with sufficient input subsets
This provides a model-agnostic explanation method for black-box decisions, addressing a key challenge in interpretable AI, though it is incremental in improving upon existing local explanation frameworks.
The paper tackles the problem of explaining black-box model decisions by proposing sufficient input subsets, which identify minimal feature sets that alone lead to the same decision, and demonstrates this method on neural networks across text, image, and genomic data, producing more concise rationales than existing techniques.
Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model. Existing techniques are often restricted to a specific type of predictor or based on input saliency, which may be undesirably sensitive to factors unrelated to the model's decision making process. We instead propose sufficient input subsets that identify minimal subsets of features whose observed values alone suffice for the same decision to be reached, even if all other input feature values are missing. General principles that globally govern a model's decision-making can also be revealed by searching for clusters of such input patterns across many data points. Our approach is conceptually straightforward, entirely model-agnostic, simply implemented using instance-wise backward selection, and able to produce more concise rationales than existing techniques. We demonstrate the utility of our interpretation method on various neural network models trained on text, image, and genomic data.