Interpretable Companions for Black-Box Models
This addresses the dilemma for practitioners who must balance interpretability and accuracy in predictive tasks, offering a flexible solution that is incremental by building on existing black-box models.
The paper tackles the trade-off between accuracy and interpretability in machine learning by introducing an interpretable companion model that can be used alongside any pre-trained black-box classifier, allowing users to choose between high-accuracy black-box predictions or slightly less accurate but interpretable rules based on input-specific needs.
We present an interpretable companion model for any pre-trained black-box classifiers. The idea is that for any input, a user can decide to either receive a prediction from the black-box model, with high accuracy but no explanations, or employ a companion rule to obtain an interpretable prediction with slightly lower accuracy. The companion model is trained from data and the predictions of the black-box model, with the objective combining area under the transparency--accuracy curve and model complexity. Our model provides flexible choices for practitioners who face the dilemma of choosing between always using interpretable models and always using black-box models for a predictive task, so users can, for any given input, take a step back to resort to an interpretable prediction if they find the predictive performance satisfying, or stick to the black-box model if the rules are unsatisfying. To show the value of companion models, we design a human evaluation on more than a hundred people to investigate the tolerable accuracy loss to gain interpretability for humans.