Concept-level Debugging of Part-Prototype Networks
This addresses the issue of improving transparency and trustworthiness in concept-based classifiers for critical applications like medical decision-making, though it is incremental as it builds on existing ProtoPNets.
The paper tackles the problem of ProtoPNets picking up confounders and shortcuts, which compromises accuracy and generalization, by proposing ProtoPDebug, a concept-level debugger that uses human feedback to fine-tune the model, resulting in outperforming state-of-the-art debuggers at a fraction of the annotation cost.
Part-prototype Networks (ProtoPNets) are concept-based classifiers designed to achieve the same performance as black-box models without compromising transparency. ProtoPNets compute predictions based on similarity to class-specific part-prototypes learned to recognize parts of training examples, making it easy to faithfully determine what examples are responsible for any target prediction and why. However, like other models, they are prone to picking up confounders and shortcuts from the data, thus suffering from compromised prediction accuracy and limited generalization. We propose ProtoPDebug, an effective concept-level debugger for ProtoPNets in which a human supervisor, guided by the model's explanations, supplies feedback in the form of what part-prototypes must be forgotten or kept, and the model is fine-tuned to align with this supervision. Our experimental evaluation shows that ProtoPDebug outperforms state-of-the-art debuggers for a fraction of the annotation cost. An online experiment with laypeople confirms the simplicity of the feedback requested to the users and the effectiveness of the collected feedback for learning confounder-free part-prototypes. ProtoPDebug is a promising tool for trustworthy interactive learning in critical applications, as suggested by a preliminary evaluation on a medical decision making task.