Prototype Generation: Robust Feature Visualisation for Data Independent Interpretability
This work addresses interpretability issues for researchers and practitioners in machine learning, offering a robust, model-agnostic tool to uncover spurious correlations and biases that quantitative test-set methods miss.
The paper tackles the problem of untrustworthy feature visualisation in image classification models by introducing Prototype Generation, a method that produces inputs with natural activation paths, and demonstrates its effectiveness through quantitative similarity measurements between generated prototypes and natural images.
We introduce Prototype Generation, a stricter and more robust form of feature visualisation for model-agnostic, data-independent interpretability of image classification models. We demonstrate its ability to generate inputs that result in natural activation paths, countering previous claims that feature visualisation algorithms are untrustworthy due to the unnatural internal activations. We substantiate these claims by quantitatively measuring similarity between the internal activations of our generated prototypes and natural images. We also demonstrate how the interpretation of generated prototypes yields important insights, highlighting spurious correlations and biases learned by models which quantitative methods over test-sets cannot identify.