ProtoShotXAI: Using Prototypical Few-Shot Architecture for Explainable AI
This addresses the need for more trustworthy and flexible explainable AI methods, particularly for users dealing with black-box models, though it appears incremental as it builds on existing prototypical few-shot architectures.
The paper tackles the problem of evaluating local interpretability in black-box neural networks by proposing ProtoShotXAI, which uses a prototypical few-shot network to explore the contrastive manifold between nonlinear features, resulting in demonstrated flexibility for model exploration on datasets like MNIST, Omniglot, and ImageNet.
Unexplainable black-box models create scenarios where anomalies cause deleterious responses, thus creating unacceptable risks. These risks have motivated the field of eXplainable Artificial Intelligence (XAI) to improve trust by evaluating local interpretability in black-box neural networks. Unfortunately, the ground truth is unavailable for the model's decision, so evaluation is limited to qualitative assessment. Further, interpretability may lead to inaccurate conclusions about the model or a false sense of trust. We propose to improve XAI from the vantage point of the user's trust by exploring a black-box model's latent feature space. We present an approach, ProtoShotXAI, that uses a Prototypical few-shot network to explore the contrastive manifold between nonlinear features of different classes. A user explores the manifold by perturbing the input features of a query sample and recording the response for a subset of exemplars from any class. Our approach is the first locally interpretable XAI model that can be extended to, and demonstrated on, few-shot networks. We compare ProtoShotXAI to the state-of-the-art XAI approaches on MNIST, Omniglot, and ImageNet to demonstrate, both quantitatively and qualitatively, that ProtoShotXAI provides more flexibility for model exploration. Finally, ProtoShotXAI also demonstrates novel explainabilty and detectabilty on adversarial samples.