Inverting Supervised Representations with Autoregressive Neural Density Models
This provides a method for researchers to analyze and quantify information flow in neural networks, though it is incremental as it builds on existing density estimation techniques.
The paper tackles the problem of interpreting features in supervised models by inverting representations using autoregressive neural density models, enabling visualization of invariances and estimation of mutual information between inputs and layers, and shows that mutual information decreases during training, supporting the information bottleneck theory.
We present a method for feature interpretation that makes use of recent advances in autoregressive density estimation models to invert model representations. We train generative inversion models to express a distribution over input features conditioned on intermediate model representations. Insights into the invariances learned by supervised models can be gained by viewing samples from these inversion models. In addition, we can use these inversion models to estimate the mutual information between a model's inputs and its intermediate representations, thus quantifying the amount of information preserved by the network at different stages. Using this method we examine the types of information preserved at different layers of convolutional neural networks, and explore the invariances induced by different architectural choices. Finally we show that the mutual information between inputs and network layers decreases over the course of training, supporting recent work by Shwartz-Ziv and Tishby (2017) on the information bottleneck theory of deep learning.