Conditional Autoregressors are Interpretable Classifiers
This addresses the need for interpretable machine learning models in domains like image classification, though it is incremental as it builds on existing autoregressive and distillation methods.
The paper tackles the problem of making image classifiers interpretable by using class-conditional autoregressive models, achieving interpretability while matching the accuracy of a standard classifier through knowledge distillation.
We explore the use of class-conditional autoregressive (CA) models to perform image classification on MNIST-10. Autoregressive models assign probability to an entire input by combining probabilities from each individual feature; hence classification decisions made by a CA can be readily decomposed into contributions from each each input feature. That is to say, CA are inherently locally interpretable. Our experiments show that naively training a CA achieves much worse accuracy compared to a standard classifier, however this is due to over-fitting and not a lack of expressive power. Using knowledge distillation from a standard classifier, a student CA can be trained to match the performance of the teacher while still being interpretable.