Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
This addresses the opacity of DNNs for researchers and practitioners, offering a method to enhance interpretability, though it is incremental as it builds on existing adversarial techniques.
The paper tackles the problem of interpretability in deep neural networks by analyzing internal representations using adversarial examples, finding that neurons respond to discriminative patches rather than semantic objects and that representations are not robust, and proposes an adversarial training scheme to improve interpretability, enabling tracing of predictions to influential neurons.
Deep neural networks (DNNs) have demonstrated impressive performance on a wide array of tasks, but they are usually considered opaque since internal structure and learned parameters are not interpretable. In this paper, we re-examine the internal representations of DNNs using adversarial images, which are generated by an ensemble-optimization algorithm. We find that: (1) the neurons in DNNs do not truly detect semantic objects/parts, but respond to objects/parts only as recurrent discriminative patches; (2) deep visual representations are not robust distributed codes of visual concepts because the representations of adversarial images are largely not consistent with those of real images, although they have similar visual appearance, both of which are different from previous findings. To further improve the interpretability of DNNs, we propose an adversarial training scheme with a consistent loss such that the neurons are endowed with human-interpretable concepts. The induced interpretable representations enable us to trace eventual outcomes back to influential neurons. Therefore, human users can know how the models make predictions, as well as when and why they make errors.