Learning Support and Trivial Prototypes for Interpretable Image Classification
This work addresses the problem of improving interpretable image classification for researchers and practitioners by offering a more accurate and interpretable model, though it is incremental as it builds upon existing ProtoPNet methods.
The paper tackles the issue of inferior classification accuracy in ProtoPNet methods by proposing a new approach to learn support prototypes near the classification boundary, inspired by SVM theory, and introduces ST-ProtoPNet, which combines these with trivial prototypes. The result is state-of-the-art classification accuracy and interpretability on datasets like CUB-200-2011, Stanford Cars, and Stanford Dogs.
Prototypical part network (ProtoPNet) methods have been designed to achieve interpretable classification by associating predictions with a set of training prototypes, which we refer to as trivial prototypes because they are trained to lie far from the classification boundary in the feature space. Note that it is possible to make an analogy between ProtoPNet and support vector machine (SVM) given that the classification from both methods relies on computing similarity with a set of training points (i.e., trivial prototypes in ProtoPNet, and support vectors in SVM). However, while trivial prototypes are located far from the classification boundary, support vectors are located close to this boundary, and we argue that this discrepancy with the well-established SVM theory can result in ProtoPNet models with inferior classification accuracy. In this paper, we aim to improve the classification of ProtoPNet with a new method to learn support prototypes that lie near the classification boundary in the feature space, as suggested by the SVM theory. In addition, we target the improvement of classification results with a new model, named ST-ProtoPNet, which exploits our support prototypes and the trivial prototypes to provide more effective classification. Experimental results on CUB-200-2011, Stanford Cars, and Stanford Dogs datasets demonstrate that ST-ProtoPNet achieves state-of-the-art classification accuracy and interpretability results. We also show that the proposed support prototypes tend to be better localised in the object of interest rather than in the background region.