But that's not why: Inference adjustment by interactive prototype revision
This addresses the issue of unreliable AI decision-making for users needing interpretable and adjustable models, though it is incremental as it builds on existing prototypical-part models.
The paper tackles the problem of AI models making predictions based on unreasonable factors due to dataset confounders, and proposes interactive prototype revision methods that allow users to correct faulty prototypes without compromising accuracy.
Despite significant advances in machine learning, decision-making of artificial agents is still not perfect and often requires post-hoc human interventions. If the prediction of a model relies on unreasonable factors it is desirable to remove their effect. Deep interactive prototype adjustment enables the user to give hints and correct the model's reasoning. In this paper, we demonstrate that prototypical-part models are well suited for this task as their prediction is based on prototypical image patches that can be interpreted semantically by the user. It shows that even correct classifications can rely on unreasonable prototypes that result from confounding variables in a dataset. Hence, we propose simple yet effective interaction schemes for inference adjustment: The user is consulted interactively to identify faulty prototypes. Non-object prototypes can be removed by prototype masking or a custom mode of deselection training. Interactive prototype rejection allows machine learning naïve users to adjust the logic of reasoning without compromising the accuracy.