Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time
This solves a problem for non-machine-learning experts like clinicians by allowing them to edit interpretable models in real-time without repeated retraining.
The paper tackles the interaction bottleneck in prototypical part models (ProtoPNets), where fixing flaws requires slow retraining, by introducing Proto-RSet to find many equally good models quickly, enabling real-time corrections while maintaining accuracy on the training set.
Interpretability is critical for machine learning models in high-stakes settings because it allows users to verify the model's reasoning. In computer vision, prototypical part models (ProtoPNets) have become the dominant model type to meet this need. Users can easily identify flaws in ProtoPNets, but fixing problems in a ProtoPNet requires slow, difficult retraining that is not guaranteed to resolve the issue. This problem is called the "interaction bottleneck." We solve the interaction bottleneck for ProtoPNets by simultaneously finding many equally good ProtoPNets (i.e., a draw from a "Rashomon set"). We show that our framework - called Proto-RSet - quickly produces many accurate, diverse ProtoPNets, allowing users to correct problems in real time while maintaining performance guarantees with respect to the training set. We demonstrate the utility of this method in two settings: 1) removing synthetic bias introduced to a bird identification model and 2) debugging a skin cancer identification model. This tool empowers non-machine-learning experts, such as clinicians or domain experts, to quickly refine and correct machine learning models without repeated retraining by machine learning experts.